Both the colon and the arrow are weird. In what sort of situations there is ambiguity in the direction of a pipe or between command and its arguments that this would solve? The colon would make more sense if redirection specifiers would be written before the colon, but alas they are not.
How does the magic with JSON etc work? The docs do mention something about mime types but that is never really explained properly. If I make my own command that outputs JSON, can I make Murex understand that without explicit casts?
Overall the whole thing feels bit quirky to me, but there is cool stuff in there too.
Thank you for your feedback. The arrow token is something I've gotten used to typing with the likes of C++ and Perl but the rational of adding it to murex was so that each command was more visually distinctive which isn't always the case with the pipe character. However I did expect it would polarise opinions so it's completely optional (ie you can use the regular pipe token if you prefer).
The colon I'm a little on the fence about myself. It's a throwback to an earlier iteration of the syntax and it does add a little "code smell" to the interpreter and code behind the command prompt. In fact I did nearly remove it from murex just last week. However it is also an optional token so you're free to ignore it if you'd prefer.
The JSON magic is just Go's standard library unmarshalling JSON into a slices and maps and then remarshalling it back to JSON at the end. This design means I can swap out a JSON interface with a YAML, TOML or most other serialisation formats at runtime. For example you can have JSON coming in then output it as YAML (or visa versa). Tabulated data such as CSV is also supported but I had to be a little more creative in how are interchanged with formats like JSON since they represent different ideas of structured data -- sadly converting between CSV and other formats is still rather buggy but I'd definitely on my road map to fix.
In terms of how a file is recognised as JSON (or any other format), there is two methods: you use the `open` builtin which is a bit like `cat` but works with HTTP URLs as well as file paths. If it's a HTTP address it uses the Content-Type header to get the MIME (just like your desktop browser would do) and passes that type information down the pipeline. If it's a local path then it checks the extension. You can see the supported extensions and MIME types by typing:
config get shell mime-types | pretty
config get shell extensions | pretty
The issue is when using a classic UNIX tool like `cat` which wouldn't know a JSON from a CSV. So to support them you can also cast if you want to. for example the following two examples do the same thing:
I completely agree there's a lot of quirkiness to the shell. Some stuff I might even do differently if I were to start again from scratch and definitely some POC code in there I really need to rewrite. This shell was born out of a desire to solve some personal annoyances with Bash and I think it's fair to say I overindulged myself in places. But I figured if I design it for myself then at least I'm guaranteed one person will use it :) Of course if anyone else likes the project and finds themselves wanting to use it then I will absolutely take their feedback on board too.
edit: also thanks for pointing out the areas where the documentation was unclear or lacking entirely. Documenting this thing has proven to be more time consuming (and less enjoyable) than writing the code! But I'll work on that.
> and passes that type information down the pipeline
That's the big question mark to me, how is the type information passed along. If I do
open foo.json | bar
then how will bar know (or will it?) that its input is json? Or is the type information only internal to the shell and not exposed to the individual pipeline parts?
It's internal to the shell however so is the individual pipeline parts to some extent.
The long answer is that all commands in the pipeline are treated as either a murex shell script or a builtin function. Any external commands are actually the builtin function called "exec" which is calling an external command as its parameters. This means I could effectively write my own pipeline with richer APIs beyond just reading []bytes of data. This doesn't come without significant drawbacks as well (which I can also talk about if your interesting) but I'm working to mitigate them.
So the following two lines of code are equivalent:
open foo.json | grep foobar
open foo.json | exec grep foobar
So if you then were to pipe that into another builtin you'd need to cast it again. eg
This is just illustrative though. Technically this would break the JSON file because "grep" just works on dumb lines and JSON isn't a linefeed document. This is why murex comes with such an extensive number of builtins, for example if you wanted to do a grep-like regex match against elements in a JSON array and output that as valid JSON you could the following pipeline:
open foo.json | regexp m/foobar/
However going back to your original question, if someone wanted to write a shell script or any other utility that support murex data-type but wasn't part of the murex shell binary, then there's no way murex could pass that type information to it.
This is a problem I've been mulling over for a few years now and I've considered a few options in that time - none of them are without their problems. At one point I even laid down some POC code but that has since been deleted because pragmatically it seems a premature problem to solve while I'm the only person developing and using the shell.
edit:
It's also worth noting that there is a default data-type that is assumed even when no data-type is specified. That type is a loosely tabulated "catch all" type format for your standard unix tools. That means output from commands like `ls -l`, `ps`, and other classic utilities can be parsed and fields selected intelligently. eg if you were to type the following partial command line then hit tab:
ps -fe | [ <tab>
You'd see the headings from `ps` in your autocompletion suggestions:
UID PID PPID C STIME TTY TIME CMD
So you could select one or more of them:
ps -fe | [ UID PID CMD ]
...and murex would filter the output of ps to only those 3 columns.
It's not perfect by any means - since there are a thousand different ways developers can output data to the terminal - but it does seem to be an asset more than a hinderance for my particular use cases. Of course YMMV and I'd welcome any feedback if that were the case :)
Not to be a spoilsport. But you may want to rethink the name.
There's a French financial software company called Murex.
Since a shell by the same name is very much computer related and I assume that they have a trademark on the name you may be in for a world of hurt, legally speaking.
Good point. I did spend a fair amount of time deciding on a name - it's surprising just how few good names aren't already being used in OSS. So I'd love some suggestions if anyone has any.
I really hate the to be the bearer of bad news, but it would really suck to invest a lot of work and when your product gets popular (and visible) to get sued.
Naming is hard, but how about something like MXShell ?
That's, of course, if it makes sense.
Edited to add: There seems to be an instagram account by that name and a Twitch streamer. Else then that I can't find much with a cursery Google search.
I genuinely appreciate the feedback. I was hoping there was enough distance between this project and the French firm but I'd stupidly not considered that they'd trademark their name.
The process isn't that interesting to be honest. I just used a crossword solver to find all the English words ending with "sh" and then Googled them to see if projects already existed.
Then I Googled different types of shells in nature to look for something that sounded interesting. That's where 'murex' came up, it's a genus of a sea snails and it also loosely followed the UNIX naming convention (Linux / Unix / Minix / etc).
I guess I could have just made a word up rather than trying to be clever :)
I like the name 'murex'. The first thing it makes me think of is murex shells - as in the sea snail - which we used to find on the beach all the time when I was a kid. And as you say the 'X' at the end conjures up Unix.
Cisco did sue Apple over the trademark for iPhone[1]. When Apple later used the name iOS they came to a private agreement with Cisco[2]. If they just used the same name without coming to terms with Cisco, they surely would have been sued again.
Wow this is really nice, I use Fish cause it's nice out of the box and not for scripting. I think that's what my sweet spot is, I use Bash for shell scripting, cause it's almost always installed or something compatible is installed in most Distros, but for my terminal emulator's shell I don't care what I use as long as it is indeed intuitive. I'll be sure to give Murex a shot.
Looks very cool. Has some of the same ideas as Nushell. Seems like a lot of these “better shells” are coming out lately, I wonder why that’s in vogue at the moment?
I don't agree with this. Or at least it's not applicable for me. I've been managing UNIX and Linux systems for around 2 decades so sadly I'm now far too old to be included as part of the younger generation any more :(
murex came about more by accident than intention. It was a personal project to "solve" a real world problem I was having at work but it kind of morphed into a shell. Even the shell aspect was very different in the beginning to what it is now. In fact almost every aspect of the shell has changed in the last 3 years.
The project is older than the Github repo because it was a private project hosted on a personal git server for the early stages of its life and I squashed the commits when uploading to Github to hide the early code (read: "mistakes") but once the project did start morphing into a shell the intention was always to make scripting a first class citizen because I specifically didn't want to use a different shell for interactive use than scripting. In fact if anything, I wanted to bring scripting paradigms to the REPL shell but without creating a command line that was too verbose for everyday use.
for a while i have been dreaming of the idea of having a real programming language for use on the commandline and i always get stuck at the verbosity of the syntax.
i didn't want to invent a new language however but possibly reuse an existing one. just that none of the languages i came across really fit.
for starters it would have to be a language that uses spaces to separate arguments, like lisp, smalltalk, red, forth and a few others. but there are more potential requirements that most languages don't really support, like being able to specify string values without quotes...
I'm with you on that one. The original murex syntax was very Javascript inspired. In fact it was even object oriented and originally called "jsh". The ironic thing is I hate JS as a language - though that's less about the syntax and more about other specific grips.
Anyhow, first I made quoting strings optional, because mandatory quotes were fucking annoying for regular use. Then the parentheses disappeared. Then the object orientated model was dropped.
Murex is a very different language now to what it was when it started; though there are still some legacy features in there which are throwbacks to early designed (eg the default data-type is called "generic", which made sense when it was OO as generic was meant to handle any content but makes less sense now that it's a first party type in its own right).
There's definitely a lot wrong with murex though. It's a long way off being a proper language. But for shell work, it has been my default daily shell for a few years now. I also use it for writing any shell scripts albeit just stuff for personal use.
From what you've posted, I'd be very interested if you ever designed a shell language. It sounds like you and I have similar desires for what the next iteration of UNIX shells should look like.
I like the data access bits, but find the arrow “operator” a bit weird. I keep wondering if a simple wrapper around jq and the like wouldn’t be enough.
Thank you for the feedback. Yes, the arrow token is definitely a polarising one. I personally like it because it's more visually distinctive but even I find myself using the pipe character accidentally on occasions due to muscle memory. So I completely understand why many others wouldn't want to use it too. This is why I made the arrow token is completely optional. You can use the traditional pipe character if you prefer. eg the following two generate the exact same ASTs:
open example.json -> foreach { -> regexp s/foo/bar/ }
open example.json | foreach { | regexp s/foo/bar/ }
You could even use both tokens in the same pipeline if you wanted:
command -> command | command -> command | command
...though I'd urge against doing that - it's better to pick one token and use that throughout your pipeline.
re jq: that certainly covers some use cases but it doesn't cover them all.
For example murex has a hint text - which is a line below the interactive command prompt - and it works a little like the tool tip text in GUIs. It's context sensitive and will give you hints about the command line you're typing such as
+ full path to an executable (including resolving symlinks) so you can be confident you're about to execute the program you think you are
+ it expands variables if any $VARs are used to give you a clue about how the final command line will be interpreted
+ it gives you descriptions about the application - parsed from man pages or the source code (if available)
+ it will display the code behind an alias so
The idea is to inform the user of any hidden traps behind shells that often lead to mistakes.
That's just the hint text. There's been some improvements in autocomplete suggestions that aren't available in most shells eg:
+ man page parsing,
+ suggestion descriptions,
+ the ability to use regexp to jump areas in a nested path (similar to fzf)
+ and it can also manages timeouts in dynamic suggestions better than bash.
I've tried to model the REPL environment of murex more on what tooling I'd expect from an IDE rather than what tooling I'd expect from a POSIX shell. That's not to say I've created a polished UI/UX nor that there isn't some Bash/zsh features missing from murex. There's still someway to go before murex will be out of it's beta stage. But that's the design inspiration behind the shell.
What is that typeface? Every vowel looks like an "o" that was cut up to look like another vowel, its absolutely hard to read. It's also ironic that there's (allegedly) a focus on readability for the language but absolutely none of that concern found its way to the website.
Thanks for the feedback. I'm dyslexic so I did actually spend a long time picking a readable typeface for the website since I often struggle with some of the artistic choices other designers inflict. I found this typeface (and the other stylings of the site) to be what was easiest for me to read.
However I do acknowledge that I haven't conducted any form of research into other people's opinions and nor have I tested the site on many other platforms aside Firefox and Safari. So what works for me might not work for other people.
To that end, if others agree it's not the most readable typeface (or any other issues with the site are pointed out) then I will definitely change it. As you rightly say, being readable is the most important concern for that site.
I've changed the typeface now, however you may have to hit refresh on your browser because I've used some pretty aggressive cache-control headers.
It's not a hugely massive change in typeface style because I really can't stand sherif when reading large chunks of text however I have also included some tweaks in the spacing which might also help.
Let me know if that has improved things or if I need a more dramatic change in typeface.
I know it can be frustrating to present work and criticism veers off to focus on other things, but thank you for being receptive. Its much better now :-)
I don't mind. I'm not a designer so I welcome any feedback on that front. I was more surprised at how bad others found it because I usually struggle with printed text.
How does the magic with JSON etc work? The docs do mention something about mime types but that is never really explained properly. If I make my own command that outputs JSON, can I make Murex understand that without explicit casts?
Overall the whole thing feels bit quirky to me, but there is cool stuff in there too.