I would highly recommend Nixops. It allows you to describe your infrastructure in the Nix expression language. Which is a proper dict-based programming language. It supports many popular clouds. I'm not sure if it is Nixos only, but Nixos itself is awesome to use with it. You just put your provisioning configuration inside your "network" configuration and boom, all your servers and hard drives are created and fully provisioned.

https://Nixos.org/nixops

A cursory look at it tells me the nix expression language isn't that much more powerful than HCL. Inheritance is nice but that's about it.

it is a fully-fledged programming lazy purely-functional programming language. Which allows you to abstract a lot of boilerplate in functions and whatnot. https://nixcloud.io/tour/?id=1

I am actually okay with not having loops or advanced constructs. Those things made using Ansible for more complex provisioning scenarios (a) unreadable and (b) horribly kludgey. m

The things that Terraform solves for, namely keeping environment state and infrastructure relationships, are really hard to do with other CM tools without diving deep i.e. writing python or ruby

It's not the lack of "advanced" constructs as you say. It is also the total disregard for all the human centuries of effort that have gone into making programming languages and environments convenient vehicles for expressing computation. Things like stack traces, syntax highlighting editors, auto-complete, interactive debuggers, and a bunch of other stuff you get with an actual programming language that you don't get with an external DSL. There is a reason those things are not part of the package. It is because making those things is extremely hard. Whereas slapping together a parser is not but feels like actual work.

You might consider those things advanced. I consider it the bare minimum to get proper work done.

Here's the thing: HCL isn't a programming language nor will it ever be. I hope. this problem isn't unique to Terraform; Chef and Puppet are in the aame boat.

It's markup. It is meant to be easily readable by devs and non-devs alike while being reasonably capable.

The beauty of these tools is that they are written and be expanded upon with popular languages. If I wanted to make Terraform deploy Dell servers, all I have to do is write a plug-in with Go. What's more is that I don't have to teach a TF user anything new since its usage pattern will be the same as any other TF plug-in.

Nope. Chef is an embedded DSL. It does not have the same shortcomings. In fact it is the only configuration management tool that approaches the problem from the right angle. Instead of re-inventing the wheel it gives me a library of composable components that I can glue together however I wish.

You must not have seen many TF deployments. I've seen several and each one is a unique snowflake. Some use Jinja, some use Makesfiles, some use Erb, some just use heroic copy-pasting. There are no conventions or standards whatsoever. I don't know what you mean by readable either. When was the last time you navigated 20 Terraform modules while mentally filling in all the variables and thought to yourself "Ya, this is totally readable".

There is also Rex that let you write Perl and use any Perl library anywhere.

> It's markup. It is meant to be easily readable by devs and non-devs alike while being reasonably capable.

Who are the non-devs reading infrastructure code? I think the target market for Terriform (probably) understands the basics of programming languages like loops, conditions, and variables.

I see your point about plugins/expansion. I think infrastructure hasn't gotten past the "write a plain file as an API" stage, so writing a library to generate those files feels like hacking on top of an incomplete platform.

With Puppet 4, Puppet is lightyears ahead of the mess that is HCL. It's not a general-purpose language, but it's strongly typed and you can work with datastructures in a functional fashion. I'll admit that some more complicated transformations are less natural than I'd like, but at least they're possible.

If I could use puppet to manage openstack instances at work, I'd use it over Terraform in a heartbeat because the foundation is much more solid, but unfortunately such a provider does not exist.

I use terraform and want to like it, but every bug and weird behaviour and nonsensical limitation is making it harder and harder. Furthermore, these aren't really fixable implementation issues. The terraform language needs a complete redesign.

You don't need explicit loops and branching to do the kind of declarative programming terraform needs, but being able to use basic datastructures besides strings would be very useful.

Terraform has lists and maps, which takes care of a number of use cases at least. Now, if you want types as opposed to data structures then you start to get into structures that are tough to serialize into JSON that is helpful for interoperability with even less featureful languages / DSLs.

Instead of if statements, pattern matching constructs would probably suffice for avoiding some of the concerns with avoiding loops and branching (limits and predictability upon the DAG and resulting state machine being generated as I understand it).

In any case, I've found plenty of power by using custom data providers that use whatever logic I want and feed that to Terraform providers and modules to instantiate. I think the level of maturity with the Terraform community is still in the early stages and emphasis will shift to data provider based constructs for orchestration.

What would you consider has the bare minimum to get the job done then? Chef? Ansible? Salt? Puppet? Some homegrown monstrosity? I have used the first two and both of them have major problems with state and they can get super complicated if you are not careful.

I think part of your issue with Terraform is that you want it to do everything, when you should just use it to get the foundation up and then use a provisioner with more power to do the finer detail work to fit your needs. Also, lets remember it's not even 1.0 software at this point. We started playing around with it around 0.6 and things have come along way since then. Maybe it's just not the tool for your needs.

You're setting up a false dichotomy. Expressing a dependency graph of cloud resources does not require a special DSL.

In fact if Terraform was a library it would not do everything nor do I want it to do everything. At the end of the day Terraform traverses a dependency graph and generates a sequence of commands to run. The traversal is basically a topological sort and the command generation is a bunch of API calls. None of those things require a specialized DSL.

And you still didn't answer my question. Based on your other comments Chef is your tool of choice for setting up servers because you can hack and glue things together how you want. That's great for you I get it. I assume you are a programmer by trade that wants to use the tools you feel comfortable with to get the job done. That's OK. I am an ops guy that is looking to make reproducible infrastructure without having to figure out awful Ruby/Chef bastardization. Terraform isn't a programming language. I understand you wish it was, but it simply isn't one. It's not the tool for someone that wants to program the infrastructure with Ruby.

I understand your point, but as a templating language the following should be expressible:

Given a list:

    [ A, B, ... N ]

Create a list of objects:

    [ 
        { foo: const, bar: A }, 
        { foo: const, bar: B }, 
        { foo: const, bar: C }, 
        ...,
        { foo: const, bar: N }
    ]

You say, why, this complicated programming task is not a job for terraform. Then why are some elements in place of the dsl functions, and some missing?

I have a feeling that the creators of may cloud tools don't use the cloud, or only to have pet infrastructures instead of pet servers. To set up something automated, complicated, scalable, reproducible, you have to result to do it yourself, while these tools are supposed to do it for you.

I feel like ops people are writing these tools for ops people, but instead of good old bash, now in Go. What is the problem with this?

* They reinvent well known patterns, but call them differently.

* They disregard the common knowledge of software development tradecraft.

* They don't get these things right for the first (or second time), just redo their previous workflows, and we are still where we were 10 years ago, just with different tools.

I personally prefer Ansible, but it has its weak points as well, and its development seems to have slowed since RedHat acquired it. The main advantages of Ansible are extensibility and reuseability. I can write custom modules, and have a descriptive DSL for stuff not thought of for the upstream devs. I can also simply reuse parts, which is a very weak point in terraform.

Still we use terraform, for it has its own merits, but IMHO if you want to do something, either do it, or don't do it, but don't do a half-assed "solution" which promises much, and fails to deliver. This is my feeling with terraform, where I could not even create a simple mapping of values, if those are not strings! (I know, "ops" people love strings. Only those pesky "devs" love structured data)

HCL is atrocious. Terraform's main issue is that it tries very hard to be declarative, then it has to invent a template language as a superset of the declarative JSON it uses to make up for it. And that superset is invariably shit.

It's like you said; no loops, no proper variables etc.

Terraform itself though is fantastic and powerful. It's lacking some pretty important stuff still (for example, not having to refresh the entire infrastructure state when you change one single variable for one single tiny little service), but that just makes it an alpha. It's still better than the alternatives by a mile and then some.

But, yeah. Fuck HCL. The good news though is that I think this is "easily" fixable by someone who would really work on it: Because the underlying environment is 100% json, it's easy enough to generate; so you could implement a saner environment for specifying the state and feed that to terraform. That's enough for me to bet my infrastructure on it: knowing that today's issues are fixable, rather than the effort being doomed from the start.

`terraform plan -refresh=false`

Granual refreshing is not the same thing as no refreshing.

`terraform refresh -target=resource_path`

I know it's possible to do manually. But you have to figure it out yourself then. It also doesn't properly dirty the other affected resources.

We've really enjoyed templating terraform. It solves a lot of the problems you're mentioning, and we can write the complex logic we need without interacting _directly_ with cloud APIs...

Compilers are like, useful.

It also seems pretty much the same as ansible (which suffers the same problems as you mentioned).

I pretty much agree with you, they should of provided 2 entrypoints IMHO, they can make there DSL and all that and make it (eventually) turing complete and at the same time they should just provide composeable objects in <some> language as well.

I've never quite figured out who the target audience is for these kind of DSL(s), is it people that can't program, or people that can't program well/at all (without adult supervision)?

I don't know either. Puppet, ansible, salt, and friends are in the same boat. The best part is that the special syntax doesn't buy them anything. They just end up being gimped languages for expressing some kind of dependency graph.

All configuration management systems suck.

They all suck in equal measure, but each in a different fashion. You can only try to pick the one tool that sucks the least for your particular use-case and hope that two years down the line you made the right choice.

This isn't much of an argument. That's like saying all programming languages suck and it's a crapshoot. It's not. There are design principles that work and scale to large scale systems and then there are things that don't.

As far as I'm concerned Chef is the right way to do configuration management. Everyone else with their YAML DSLs is doing it wrong. Expressing dependencies between resources does not require specialized external DSLs and sometimes you need to do imperative things to get to the end which is either impossible or just horribly convoluted with any tool that tries to hide complexity behind a markup/serialization language.

Apparently these people think it makes it "simpler". Well I'm sorry but me looking up how to format things with yaml and no autoformat and no intellisense just isnt good and doesnt make things simple. Even xml with a backing xsd would be better than that, but that's apparently not cool enough anymore.

I forgot about the lack of auto-complete and the horrible error messages. At least with a library when I have a syntax error or something blows up I have a debugger and the actual language at my disposal to figure stuff out. No such things are remotely available with all these external DSLs. If even there is any semblance of error reporting.

Several human centuries of effort have been spent on improving these things (syntax highlighting, auto-complete, stack traces, error messages, interactive debuggers, etc.) and then all these DSL authors decide to just chuck it all out. Boggles the mind.

More than a few times Terraform has wrapped an error message from AWS in the hopes of being helpful while hiding all the details of the actual error message and no hints how to rectify things in the DSL.

Syntax highlighting and auto complete is available for HCL (The language used by Terraform)

e.g.

https://plugins.jetbrains.com/plugin/7808-hcl-language-suppo...

https://github.com/wholroyd/vscode-hcl

Which is great and all but like the op said our files are actually jinja2 templates because HCL just doesn't cut it.

I had the same experience with ansible. It also wraps the responses in some ultra verbose json. I really just want to see the remote commands being executed, if you return them wrapped in some kind of format then you need to provide a gui for it. Otherwise it's all worse than before.

Would you know a recent tool based on XML + XSD (or RNG) in the space (I believe IBM had one like 10 years ago, but it got sacked because XML)? I've never been a fan of using markup languages for things other than text/semistructured data, but yaml/toml/json is such a huge step backwards that I'm having trouble recommending it.

Some basic conditional logic was added in 0.8 [0]. The reliance on 'interesting' declarative tricks, e.g. counts for conditionals and loops, is a little annoying I agree. However, it looks like they're listening to the community and will build on these features in the future. There's a lot to work on :)

0: https://www.hashicorp.com/blog/terraform-0-8/#conditional

Take a look at jsonnet: http://jsonnet.org. It extends JSON with loops, conditionals, variables and more in an elegant way. We don't write JSON anymore and treat it like an "assembly language." We use jsonnet to generate CloudFormation templates, terraform templates and Kubernetes resource definitions.

Hear, hear! Devops right now is in the Middle Ages, kind of like what web frontend programming was a few years ago. Remember all those weird cobbled down logicless DSLs that were supposed to save us from ourselves.

I am expecting the Renaissance you mentioned in devops shortly, just like how React transformed the scene.

In short, DSLs should be inside programming languages and we shouldn't be afraid to write code.

But this isn't a programming language. it's a DSL that lets you get the job done. I get that you want to use flow control statements to make your life easier, but I feel that part of the reason why Terraform was made and has gained a lot of popularity. For me, it doesn't make things overly complicated. Except for how they handle variables with modules.That stuff sucks.

I don't really agree that Terraform re-invents wheels. In fact it feels like a lot of the things you cite as issues are actually due to Terraform being pretty innovative. There's not much like it for "version-controlled" infrastructure, so the team and community are learning as they go, and can't be expected to come up with perfect solutions right away. I agree the looping hacks are very ugly, really dislike the whole split and join everywhere thing.

As for DSL vs. programming languages, the halting problem provides a pretty compelling argument in favor of limiting your system if you want to have "verifiable convergence". Personally I'm a huge fan of HCL too. It fixes a lot of what's bad about YAML and TOML and has a built in pretty printer.

You should try out http://jsonnet.org, I love it.

Ahh just like Ansible with it's YAML'ish templating language. If you want to add variables and scoping, loops, a module resolution system, etc., then I'd rather use a real programming language.