The Year the Cloud Fell (The Fallen Cloud Saga Book 1)

---

It is not a given that cloud providers must offer low-latency interconnects more widely, but if they make the business decision not to do so, they must recognise that there will always be a segment of the market which is closed to them.

See a Problem?

Rather than trying to bluff their way into the high-end HPC market, cloud vendors who choose to eschew the low-latency segment should focus on their genuine strength; the near-infinite scale they can offer for high-throughput workloads and cloudbursting of single-node applications. Before we even reach the complexities of managing data once it is in the cloud, there are issues to be faced with getting it there, and eventually getting it back. At the moment then, there is no obvious reason to favour one provider over the others on this front. For industry users, data being held hostage as it grows in volume is less of a concern — the chain of ownership is much more straightforward, and as long as the company retains an account with the cloud provider, someone will be able to access the files whether they are in a position to make decisions about data migration is another question….

Data produced by university researchers is more tricky in this regard — funding council rules are deliberately non-specific about what is actually required from researchers when they make a data management plan. The general consensus is that published data needs some level of discoverability and cataloguing; implementing a research data service in the cloud is likely to be far easier in the long-term than providing an on-premise solution, but requires a level of commitment to operational spending that many institutions would not be comfortable with.

Cloud providers could certainly afford to make this easier. The storage landscape within the cloud presents another complication, one which many HPC users will be far less prepared for than simply tuning their core-count and wall-clock times.

NSF Extends Access to Its Leadership Systems Blue Waters & Frontera

Migrating data directly in and out of instance-attached block storage volumes via SSH might be the way to go for short, simple tasks — but any practical workflow with data persisting across jobs is going to need to make use of object storage. While the mechanisms to interact with object storage are fairly simple for all three cloud providers, the breadth of options available when considering what to do next stick with standard storage, have a tiered model with migration policies, external visibility, etc could lead to a lot of analysis paralysis.

For researchers who just want to run some jobs, storage is the first element of the cloud they will touch which is likely to provoke a strong desire to give in and go back to waiting for time on the local cluster. For more demanding users, the problems only get worse — none of the built-in storage solutions available across the public cloud providers is going to be suitable for applications with high bandwidth requirements. Parallel file systems built on top of block storage are the obvious fix, but can quickly become expensive even without the licensing costs for a commercially supported solution.

Managing high-performance storage on an individual level is going to require more heavyweight automation approaches than many HPC researchers will be used to deploying, and so local administrators could suddenly find themselves supporting not one, but dozens of questionably optimised Lustre installs.

The Year the Cloud Fell

A parallel file system appliance spun up by the cloud provider is the obvious solution here — just like database services and Hadoop clusters, the back-end of a performant file system should not need to be re-invented by every customer. Much of the ecosystem associated with each public cloud is predicated on the fact that third-party software vendors can come along and offer a tool which manages, or sits on top of, the IaaS layer. These third parties then charge a small per-hour fee for use of the tool, which is billed alongside the regular cloud service charges.

Alternatively, a monthly fee for support can be used where a per-instance charge does not scale appropriately.

Kurt R. A. Giambastiani

These models both work pretty well for enterprise, but do not mesh well with scientific computing, which is typically funded by unpredictable capital investments — a researcher with a fixed pot of money needs to be really confident that your software is worth the cost if they are going to adding a further percentage on top of every core-hour charge they pay. More often, they will choose to cobble something together themselves. This duplication of effort is a false economy as far as the whole research community is concerned, but for individuals it can often appear to be the most efficient way forward.

Cloud providers could address the low-hanging fruit here by putting together their own performance-optimised instance images for HPC, based on for example a simple CentOS base and with their own tested performance tweaks pre-enabled, hyperthreading disabled, and perhaps some sensible default software stack such as OpenHPC. Doing this themselves, rather than relying on a company to find some way to monetise it, would give the user community confidence that their interests are actually being taken into consideration.

In order to be cost-competitive with on-premise solutions, cloud HPC requires the use of pre-emptible instances and spot-pricing. Once that happens, the lessons learned from the migration can filter out to the rest of the industry.

Navigation menu

The challenges of funding an open-ended operational service out of largely capital-backed budgets are a barrier to wholesale adoption of the cloud by universities, though this is one which central government really ought to be the ones to address. Cloud vendors can certainly help matters — the subscription model taken by Azure is a good start, but needs to be rolled out to the other providers and explained much better to potential users.

Finally there is, perhaps, scope for these multi-billion dollar companies to accept some of the cost risk by allowing for hard caps on charges or refunds on a portion of pre-empted jobs, mirroring the way that hardware resellers are expected to cope with liquidated damage contract terms. Call it a charitable donation to science and they might even be able to write it off…. Cloud providers have a few ways to get out of the doldrums they currently find themselves in with regards to the HPC market. Firstly, they should sanitise their sign-up process; AWS has this covered for the most part, but the Windows-feel of Azure is surely off-putting to hardcore technical users.

GCP offers probably the most comfortable experience for this crowd, but desperately needs to do something about the fact that individuals trying to sign up for a personal account in the EU are warned that for tax reasons, the Google cloud is for business use only; I hate to think how many potential customers have been dissuaded from trying out the platform based on this alone. Secondly, they need to find a way to be more open-handed with trial opportunities suitable for research computing.

The standard free trials available for AWS, Azure and GCP are generous if you are an individual hosting a trove of cat pictures, but not so much when you are dealing with terabytes of data and hundreds of core-hours of usage. D rated it it was ok Nov 04, Charles rated it it was ok Jan 26, Kathy Rippel rated it it was ok Sep 05, Josh rated it really liked it Jan 26, Chuck rated it it was ok Jul 29, Sara Silver rated it it was amazing Jun 23, Steve Staab rated it it was ok Apr 03, John Alexander rated it liked it Aug 12, Mark Stone rated it really liked it Jul 17, Maurynne Maxwell rated it really liked it Jun 07, Joe rated it it was ok Jan 02, Kaylee Anderson rated it it was amazing Jun 10, Cassandra Phoenix rated it really liked it Mar 12, Rigel rated it really liked it May 02, Alex rated it really liked it Jan 15, James Chapman rated it it was ok May 06, James Myers rated it it was amazing Jan 31, There are no discussion topics on this book yet.

Giambastiani is the author of nine novels, including the acclaimed alternate history series, The Fallen Cloud Saga. He is known for blending genres in his novels, and for strong historical and cultural detail. He played viola for many years in orchestras and chamber groups, until h Kurt R.

Accessibility links

He played viola for many years in orchestras and chamber groups, until he took up writing in the early '90s. Kurt now lives in Seattle, where he works as a software developer and blogs about writing and living in the Pacific Northwest. When not working or writing, he likes to cook, garden, and repair antique clocks and pocketwatches.

Other books in the series.

The Fallen Cloud Saga 5 books. Books by Kurt R. Trivia About The Year the Clou No trivia or quizzes yet. Just a moment while we sign you in to your Goodreads account. There are 10 main groups of clouds called genera that are divided into three levels — cloud low, cloud medium and cloud high — depending on the altitude at which they are found.

The height and temperature of the atmosphere where the clouds exist will determine whether they are made up of ice or water droplets, and ultimately how much they weigh.

Mammatus breast in Latin clouds are not clouds in their own right but a side effect of other clouds, and look particularly impressive with cumulonimbus clouds. Composed mainly of ice, they often form on the back edge of a retreating storm. Stratus clouds are mid-level layer clouds and are fairly boring to look at, mostly producing a flat, grey, overcast sky.

Being so thin they only tend to produce light drizzle or snow. Scattered cumulus clouds showing little vertical growth on sunny days used to be termed fair weather cumulus. As they grow vertically, they become towering cumulus and with strong updrafts they can become deep cumulonimbus clouds producing rain. Cumulus clouds can develop and grow into towering cumulonimbus clouds with cloud tops extending to more than 39,ft.

These clouds contain vast amounts of energy and water vapour which can develop into heavy rain, lightning and even tornadoes. Cirrus clouds are found at high altitude around ft where their thin and wispy appearance is shaped by strong winds. Due to cold temperatures, the clouds are mainly composed of ice crystals.

Cirrus have also been observed on other planets. Altocumulus clouds are mid-level clouds that are sometimes referred to as a mackerel sky. The clumps of cloud known as cloudlets resemble cotton wool balls and can indicate storms if the tops are bumpy, meaning that the air higher up is unstable.

• Reminiscences of Scottish Life and Character
• Regard de femme (Policier) (French Edition)
• Understanding Sarbanes-Oxley, What is different after June 2013
• La rivelazione dellantica carta (Leggereditore Narrativa) (Italian Edition)
• Chess for Kids: How to Play and Win