Microsoft's timing was the worst. I think they announced this a year after Swift was already announced, but before Swift was open sourced. So Microsoft wouldn't be able to deal with things like Obj-C/Swift interop which iOS developers were already jumping onto. And Microsoft's Windows 8 mobile initiative was pretty clearly a flop by this point.
Frankly, this Obj-C effort needed to be done way earlier, starting with AppKit, like back when Microsoft was panicking that OS X 10.4 Tiger was going to kick Longhorn's butt. If these tools had already been proven useful before the dawn of the iPhone, Microsoft might have had a chance of riding the iOS wave.
declarations for global variables
for-loop variables are read only
floats are printed in decimal with enough digits to be read back correctly.
more levels for constructors
table.create
utf8.offset returns also final position of character
external strings (that use memory not managed by Lua)
new functions luaL_openselectedlibs and luaL_makeseed
major collections done incrementally
more compact arrays (large arrays use about 60% less memory)
lua.c loads 'readline' dynamically
static (fixed) binaries (when loading a binary chunk in memory, Lua can reuse its original memory in some of the internal structures)
dump and undump reuse all strings
auxiliary buffer reuses buffer when it creates final string
>> I really wish the Lua authors would add official types to Lua.
> Never going to happen IMO. Adding static types would change the nature of the language completely, even more than it has in Python.
You both are kind of right.
The Lua authors have been working on the new companion language to Lua named Pallene. Pallene is a subset of Lua that adds types, not for the sake of types themselves, but for the purpose of performance. The Pallene compiler can generate optimized native code that potentially removes the need to manually write a module for Lua in C.
The other cool trick is that Pallene and Lua are completely interoperable with each other, so Pallene can be added to existing Lua projects, and you can opt to use regular Lua for the dynamic parts of your code where compilers won't be able to optimize much and strong types might be more trouble than help.
Well that's kinda disappointing. I expected something more in 2025, like directly generating asm like a lot of languages are starting to do.
And your article makes it ambiguous whether it's from the Lua authors or grad students. I assume it started out just the students and then the Lua authors joined in?
One of Lua's goals has been extreme portability, and the main implementation works on anything that has a C compiler, going to the extreme of compiling cleanly on C89, C99, and even compiling as C++ (no extern "C"). Remember that Lua is popular in the embedded space too, so this is a big feature.
Pallene isn't designed to be a new native language on its own. Pallene is designed to be a companion language for Lua, specializing in a subset of performance.
But as importantly, Pallene isn't just compiling to C. Pallene is generating C code that directly manipulates the underlying Lua internals, which are in C.
The research thesis is that many bottlenecks are due to boxing and unboxing going through an FFI. Memory safety also incurs overhead. Python is an extreme example of how excruciatingly slow this can be, but even Lua incurs costs for this. A core tenant of the Pallene compiler is that it can generate C code that gets to cheat like crazy. Pallene gets to directly access Lua internals and things like arrays directly manipulate underlying C arrays deep inside, which sidesteps boxing/unboxing. The compiler can do the analysis to make sure it doesn't cheat in a way that is unsafe. Finally, the C optimizer now also has a chance to perform optimizations. And now operations such as crunching math on arrays of numbers may get much faster because now you get generated code that is more CPU friendly and may benefit more from prefetching and cache locality.
Pallene is built from the the extreme compatibility goals as Lua since it is designed to work with it. It it only depends on any C compiler and Lua itself. If you get Lua compiled, then you can get Pallene working. That means any existing project that uses Lua (5.4) could start adding Pallene modules to their project for new features or to try to improve performance in key areas. Since Pallene just outputs a Lua modules, it looks like any other Lua module implemented in C, so it won't create new portability constraints that you didn't have before. This is different than say LuaJIT, where not all platforms may allow JIT or you may be targeting a new CPU architecture that LuaJIT does not support.
Both Teal and Pallene were started by grad students of Roberto's. Since Roberto has started giving talks himself on Pallene, I'm assuming they are joining in.
I'm Pallene's lead maintainer. Currently the code is maintained by me, my students and other open sou rce collaborators. We collaborate with Roberto over some Pallene-related research, specially about the type system semantics, but he isn't an active committer.
If it goes further and generates native C control flow statements when possible ("if", "for", native functions, native function calls, etc), I think it could be an omni-level language, generating basically Lua statements when dynamic and C when not, and mixing them all within the same program, entirely controlled by how much type information you give it (and how you use tables and metatables).
Alas, as of last month we changed Pallene's compiler and it now generates C gotos for control flow. ( ^ _ ^ メ ) It helped with certain optimizations...
We switched from a hierarchical internal representation to a flat one, with gotos. Made it easier to write textbook otimization algorithms. In theory we could try to reconstruct the if and while statements when it's time to emit C, but gotos are more straightforward now.
This sounds like it could compete with V8 at some point in the next few years. Honestly it could be a game changer. Looking forward to watching its progress.
This is a separate thing with the same name. Although both share some common ideas. The grimfang4/sdl-gpu is a separate library used with SDL, while the new SDL GPU API is directly part of SDL. grimfang4/sdl-gpu is much older and works with today's SDL 2.
The grimfang4/sdl-gpu was one good way to take advantage of modern GPUs in a simple way and workaround the holes/limitations of the old SDL 2D API. The new SDL 3 GPU API will likely make the need for things like grimfang4/sdl-gpu redundant.
The old SDL 2D API was not powerful enough. It was conceived in the rectangle sprite blitting days, when video hardware was designed very differently and had drastically different performance characteristics. If you wanted anything more, OpenGL used to be 'the best practice'. But today, the landscape competes between Vulkan, Metal, and Direct3D, and hardware is centered around batching and shaders. Trying to target OpenGL is more difficult because OpenGL fragmented between GL vs. GLES and platform support for OpenGL varies (e.g. Apple stopped updating GL after 4.1).
A good example demonstrating where the old SDL 2D API is too limited is with the 2D immediate mode GUI library, Nuklear. It has a few simple API stubs to fill in so it can be adapted to work with any graphics system. But for performance, it wants to batch submit all the vertices (triangle strip). But SDL's old API didn't support anything like that.
The reluctance was the SDL maintainers didn't want to create a monster and couldn't decide where to draw the line, so the line was held at the old 2D API. Then a few years ago, a user successfully changed the maintainers' minds after writing a demonstration showing how much could be achieved by just adding a simple batching API to SDL 2D. So that shifted the mindset and led to this current effort. I have not closely followed the development, but I think it still aims to be a simple API, and you will still be encouraged to pick a full blown 3D API if you go beyond 2D needs. But you no longer should need to go to one of the other APIs to do 2D things in modern ways on modern hardware.
I think you are getting confused between SDL_Render and SDL_GPU. SDL_Render is the old accelerated API that was only suitable for 2D games (or very primitive looking 3D ones). SDL_GPU is a fully-featured wrapper around modern 3D APIs (well, the rasteriser and compute parts anyway, no raytracing or mesh shaders there yet).
I was referencing the historical motivations that led to where we are today. Yes, I was referring in part to the SDL_Render family APIs. These were insufficient to support things like Nuklear and Dear ImGui, which are reasonable use cases for a simple 2D game, which SDL hoped to help with by introducing the SDL_Render APIs in SDL 2.0 in the first place.
One day, a valid argument was made that basic 2D triangles are pretty powerful in themselves for not much more code, and it notably makes wiring the excellent Dear Imgui library to an SDL app nice and clean. Even here I was ready to push back but the always-amazing Sylvain Becker showed up not just with a full implementation but also with the software rendering additions and I could fight no longer. In it went.
The next logical thing people were already clamoring for back then was shader support. Basically, if you can provide both batching (i.e. triangles) and shaders, you can cover a surprising amount of use cases, including many beyond 2D.
So fast forwarding to today, you're right. Glancing at the commit, the GPU API has 80 functions. It is full-featured beyond its original 2D roots. I haven't followed the development enough to know where they are drawing the lines now, like would raytracing and mesh shaders be on their roadmap, or would those be a bridge too far.
From what I understand, they are only going to support features that are widely supported and standardised. Thus, even bindless didn't make the cut. Raytracing, mesh shaders, work-graphs, etc. almost certainly won't make it until SDL4 10 years from now; but I am not part of the development team, so don't quote me.
Does SDL3 still use integers for coordinates? I got annoyed enough by coordinates not being floating point in SDL2 that I started learning WebGPU, instead. This was even though the game I was working on was 2D.
The issue is, if you want complete decoupling (in the sense of orthogonality) among all four of:
- screen (window) size & resolution (especially if game doesn't control)
- sprite/tile image quantization into pixels (scaling, resolution)
- sprite display position, with or without subpixel accuracy
- and physics engine that uses floating point natively (BulletPhysics)
then to achieve this with integer drawing coordinates requires carefully calculating ratios while understanding where you do and do not want to drop the fractional part. Even then you can still run into problem such as, accidentally having a gap (one pixel wide blank column) between every 10th and 11th level tile because your zoom factor has a tenth of a pixel overflow, or jaggy movement with wiggly sprites when the player is moving at a shallow diagonal at the same time as the NPC sprites are at different floating point or subpixel integer coords.
A lot of these problems could be (are) because I think of things from bottom up (even as my list above is ordered) where a physics engine, based on floating point math, is the source of Truth, and everything above each layer is just a viewport abstracting something from the layer beneath. I get the impression SDL was written by and for people with the opposite point of view, that the pixels are important and primary.
And all (most) of these have solutions in terms of pre-scaling, tracking remainders, etc. but I have also written an (unfinished) 3D engine and didn't have to do any of that because 3D graphics is floating point native. After getting the 2D engine 90% done with SDL2 (leaving 90% more to go, as we all know), I had a sort of WTF am I even doing moment looking at the pile of work-arounds for a problem that shouldn't exist.
And I say shouldn't exist because I know the final output is actually using floating point in the hardware and the driver; the SDL1/2 API is just applying this fiction that it's integers. (Neither simple, nor direct.) It gets steam coming out my ears knowing I'm being forced to do something stupid to maintain someone else's fiction, so as nice as SDL otherwise is, I ultimately decided to just bite the bullet and learn to program WebGPU directly.
No, they added float versions for most functions and I think they plan on deprecating the int API in the future. The only exception I can think of offhand is still needing an integer rect to set a viewport.
I was messing around a bit with SDL2 and either I was doing something wrong or it was just plain slow. My machine is plenty fast, but even just blitting a few dozen PNGs around a screen 60 times a second was pushing its limits. I freely admit I may have been doing something wrong, but I was surprised at just how inefficient it was at a task that we used to do without too much trouble on 1Mhz CPUs.
Maybe SDL_RenderCopy is the wrong API to use to blit things from a sprite sheet onto a display? The docs didn't give any warning if this is the case.
How recent a version were you using? Plenty of games and graphical apps use SDL2 under the hood, and rendering rects from a spritesheet is trivial. Recent versions use the geometry API for rendering rects, so it should be able to handle tons of sprites without much effort.
I'm using SDL2 2.30.0. The main loop is pretty simple, it does a few SDL_RenderFillRects to create areas, then several SDL_RenderCopy where the source is a SDL_Texture created from a SDL_Surface using SDL_CreateTextureFromSurface that was loaded from files at boot. A final call to SDL_RenderPresent finishes it off. They do include an alpha channel however.
I was expecting the sprite blitting to be trivial, but it is surprisingly slow. The sprites are quite small, only a few hundred pixels total. I have a theory that it is copying the pixels over the X11 channel each time instead of loading the sprite sheets onto the server once and copying regions using XCopyArea to tell the server to do its own blitting.
This should be plenty fast. SDL_RenderCopy generally should be doing things the 'right' way for on any video card made roughly in the last 15ish years (basically binding a texture in GPU RAM to a quad).
You probably need to due some debugging/profiling to find where your problem is. Make sure you aren't creating SDL_Textures (or loading SDL_Surfaces) inside your main game play loop. You also may want to check what backend the SDL_Renderer is utilizing (e.g. OpenGL, Direct3D, Vulkan, Metal, software). If you are on software, that is likely your problem. Try forcing it to something hardware accelerated.
Also, I vaguely recall there was a legacy flag on SDL_Surfaces called "hardware" or "SDL_HWSURFACE" or "SDL_HWACCEL" or something. Don't set that. It was a a very legacy hardware from like 25 years ago that is slow on everything now.
Whatever the problem is, it probably isn't SDL. Here's a test project I worked on[0], and I'm using a garbage laptop. The sprites aren't that big but if you're just using a single texture it shouldn't matter, since SDL does sprite batching anyway.
Your theory might be right - the first thing I would look for was something allocating every frame.
As another extremely simple idea, in my personal experience, just using regular Lua table syntax for serialization and then pre-compiling it with luac so it could be loaded quickly via dofile(), produced results just as fast for loading it in as using lua-protobuf. (I don't remember the difference between writing out Lua tables vs. lua-protobuf because my use case needed to read the data more often than generate it, but it must have not been big enough, if any, otherwise I would probably remember it.) I was loading gigabytes of data for large batch processing.
We exactly did the first half in my previous job, because JSON was really painful to use as is (and none of JSON5 or HOCON or whatever were popular enough at that time). The second half was substantially different due to our idiosyncratic requirements though.
For what it’s worth, luac has been deprecated now.
[EDIT] sorry folks! Must have misunderstood something I read a while back. Trying to dig it up now. But I could’ve sworn I’d read somewhere that this at least wasn’t suggested, and I thought it was also removed from the build in a 5.4 patch. Will circle back if I find what I’m looking for.
Where did you see that? I'm skeptical of that claim because I know some embedded uses of Lua strongly utilize pre-compiling to Lua bytecode so they can keep their hardware as slim as possible. I know some will even strip out the Lua compiler from the binary they ship to shrink the profile even more. Also, many video games like to pre-compile their shipped Lua scripts to reduce their load times.
I know that the Lua team has internal private repositories, and that luac is developed in a separate repo for them. I've seen occasional reports on the Lua mailing list that luac.c is forgotten or not updated in non-official releases. That is because those source drops didn't go through the full official release process which includes merging in from their luac repo. Maybe you are confusing these intermediate source drops with deprecation? If there is deprecation, I would like to see the details on that. I presume they would be introducing some kind of replacement that addresses all the real world use cases that rely on the abilities of pre-compiling to Lua bytecode.
> But I could’ve sworn I’d read somewhere that this at least wasn’t suggested
Maybe you’re thinking about the use of untrusted bytecode? Loading it is strongly discouraged because that’s insecure - Lua has nothing like a bytecode verifier.
Are you asking if there are special numbers or strings or some other kind of plain data that would glitch out the interpreter when they're read back in? That would be an impressive failure of a programming language.
Yes it's safe as long as you're serializing correctly (which isn't very hard).
By definition you can't trust that "untrusted data" has been serialized correctly.
Lua has strong sandboxing capabilities (i.e. ability to limit and control the environment visible from a chunk of code), but the Lua authors years ago explicitly disclaimed the ability to sandbox untrusted code. The compiler and runtime are not bug free. They don't have the resources, notwithstanding that compared to any other non-formally verified implementation their track record is pretty decent, even compared to past and current efforts from Sun, Microsoft, Mozilla, and Google. If you want to run untrusted Lua code, Lua sandboxing should be just the first of multiple line of defense, just as modern web browsers rely on various operating system mechanisms to constrain breakouts.
> By definition you can't trust that "untrusted data" has been serialized correctly.
Please read the comment again. They said the person loading the data is the same person that serialized it. The data came from an untrusted source prior to being serialized.
For example, consider a guestbook program. People send it untrusted text and then the program serializes it into a database. Reading the database back is safe.
By definition you can't trust that "untrusted data" has been serialized correctly.
Lua has strong sandboxing capabilities (i.e. ability to limit and control the environment visible from a chunk of code), but the Lua authors years ago explicitly disclaimed the ability to sandbox untrusted code. The compiler and runtime are not bug free. They don't have the resources, notwithstanding that compared to any other non-formally verified implementation their track record is pretty decent, even compared to past and current efforts from Sun, Microsoft, Mozilla, and Google. If you want to run untrusted Lua code, Lua sandboxing should be just the first of multiple line of defense, just as modern web browsers rely on various operating system mechanisms (process separation, filesystem to constrain breakouts.
> IMHO there wasn’t any reason to bother fiddling with the definition of BOOL.
There were good reasons to change this to a real boolean. I think it was a WWDC, where kind of discussing this topic, one Apple engineer said they did an audit of production Obj-C code and found multiple instances where their BOOL values were values other than 0 or 1 (which was considered surprising and ultimately bugs).
This can lead to all sorts of subtle problems. One common one encountered by framework users is where something returns a BOOL and the user explicitly compared against == YES or == NO. If the value is say 2, then this code usually results in wrong decision.
As a historical note, this was also a thorny issue for language bridges to Objective-C, going beyond just Obj-C++. I'm thinking of bridges like PyObjC, RubyCocoa, CamelBones, and LuaCocoa.
BOOL was one of those things that would have been really nice to be a real bool because when bridging to other languages that had a real boolean type, the idioms could be completely natural when writing in the bridged language. But since BOOL was a 'signed char', using only the Obj-C runtime introspection,
this by default would typically bridge to something weird like an integer.
In Mac OS X 10.5, Apple officially supported PyObjC and RubyCocoa, and introduced "BridgeSupport" which provided additional metadata and dylibs containing symbols for inline functions, so any language could create a full bridge to Cocoa. The metadata could be used to interpret if a 'signed char' could actually be used as a boolean or not. But BridgeSupport was not available for 3rd party APIs (unless the authors decided to support it, which was almost never).
There were bug requests filed for Apple to properly redefine BOOL to a real boolean for the 64-bit Mac ABI, before Apple had finalized it, but Apple didn't fix this in time. My memory is hazy, but I think when the iPhone ABI came around, they didn't have time to address this. So it would be another full arch ABI before there would be a chance to address this again.
I didn't comprehensively investigate the issue to learn whether this reflected a more systematic change in reported types, but FWIW a code comment from some Lua bridging code:
-- 2022-12-30: On macOS 12/x86_64 methods returning BOOL return integers
-- from objc.msgsend as the type encoding uses the 'c' code for char rather
-- than 'B' for _Bool (C) or bool (C++). But on macOS 11/arm64 the type
-- encoding is 'B' and we get a boolean return type.
local function itob(v, err)
if isinteger(v) then
return v ~= 0
elseif isboolean(v) then
return v
else
return error(err or sformat("expected boolean or integer, got %s", type(v)), 3)
end
end
I saw a few presentations from the 280 North guys. It was clearly impressive what these 3 guys built. Learning from Cocoa/Objective-C/Interface Builder well, they realized they needed to build 3 separate, but interconnected things, the framework (Cappuccino), the language (Objective-J), and the GUI builder (Atlas).
They sold to Motorola for $20 million in 2010, and that was the kind of the last I heard.
The food, including Cajun & Creole, crawfish, The French Quarter, Mardi Gras, voodoo, bayous, the music (birthplace of jazz), Louis Armstrong, Fats Domino, Jerry Lee Lewis, salt domes/caverns (which is why much of the US Strategic Petroleum Reserve is located there), the template for haunted mansions and graveyards.
Frankly, this Obj-C effort needed to be done way earlier, starting with AppKit, like back when Microsoft was panicking that OS X 10.4 Tiger was going to kick Longhorn's butt. If these tools had already been proven useful before the dawn of the iPhone, Microsoft might have had a chance of riding the iOS wave.