I have had enough of procedural imperative programming languages, and the retarded computer architectures that foster them!
How can you write x = x + 1; without feeling like an idiot?
The current insane state of affairs is all based on the upside down idea of computers invented by Johnny Von Neumann. It’s all about the CPU (and please be sure to genuflect reverently as you read CPU), which modifies the state of some memory (spit!). For special obfuscation, please find the rules for how said state is to be modified or evolved somwhere in the same memory.
To draw the hierarchy out explicitly, it is:
CPU –> Memory
It’s oh so easy in hindsight to cast aspersions on the genius of the man (and to be accurate, there were other contributors to the concept), but this idea is truly misguided. Like all good mind crippling paradigms, it leads directly to a number of BIG problems, the solution to which is naturally MORE of the same mind crippling paradigm.
I’ll just mention 3 here:
- The only way to know the result of a program is to run the program. It’s no coincidence that this seems to be the preferred programming paradigm of the borderline (or fully) autistic, as this seems to be the way they think anyway.
- You end up with all the state moving through a “von Neumann Bottleneck” between the cpu and the memory.
- Because state changes with time, you need to explicitly manage the time order of processing. This means parallelism is hard, and the natural way to speed things up is to speed up the CPU and the memory interface.
Of course, to a certain class of hardware guy this is the Right Thing. Like the engine freaks and their modified cars – you know, heaps of power or torque, but actually undriveable in a real word scenario – it’s all about the grunt of the CPU.
After all, when your program is a) do this tiny step, b) then do that tiny step, c) then do the next tiny step, etc… it all comes down to how fast can you “do a tiny step”. That means a processing beast, and of course, to feed the beast and keep it stoked you need a Hard Core memory interface. Just look at the memory interface of a modern computer. Level after level of caches, prefetchers, burst fills, speculative executions – its a hardware guy’s wet dream.
And please don’t mention Harvard architecture. I mean, one memory interface? Not enough! Several memory interfaces? Too many! Exactly TWO memory interfaces? Just right! (this insightful concept of computer engineering first espoused by one Goldilocks, I believe).
OK, enough griping. What is my alternative?
Well, for a change, think about the computer the other way up. With this world view, it’s all about the memory. That is, RAM, or disk, or paper – whatever.
Think about it: with a GUI, you already look directly at a big chunk of the computer’s memory on a visual display, and “see” the resulting state of a computation. Or on disk, you have the results of a computation in a file. So the memory is the most important thing.
Now, you need to modify the state of the memory. Think in terms of data flow: Memory contains data. Data flows out of the memory, is acted upon, and results (data) flow back into the memory.
That leaves the last question: How is data acted upon? That would be by a processor. It could be a CPU, but it could also be a piece of dedicated, or reconfigurable hardware (for example, an mpeg decoder). Whatever it is, think of it as a *function*.
See, this hierarchy is the exact opposite of the von Neumann:
Memory –> Data –> Processors
This is not good for the CPU designer’s sense of self importance – he’s relegated to just one of potentially many catch bags on the side of the memory. But it is a good way to solve the big issues, viz:
- The final result of a computation will be the evaluation of functions. If the functions f and g are correct, you don’t need to run the program to know the result g(f(x)) will be correct.
- Instead of one, super-duper interface to one mega CPU, you can have many interfaces to many functional processors, and those processors can be optimized for the function they perform.
- The functions (processors) have no state, just result = function(input). Time is not relevant. The result of the function does not change with time. Of course it takes time to produce the result, but that can be considered a quantum, a tick. Start a computation, *tick*, result is available. So now you can do parallelism, with a few simple primitives like composition, split, join, rendezvous, etc.
So now I think you can guess what my wet dream computer architecture looks like:
Memory.
Many interface pipelines to dynamically configured processing elements. Configured with a Hardware Description language. Could be verilog, could be VHDL, could be something more radical (for example, check out Bluespec). Whatever. But please, no more x86 brain damage CPUs.
Computations programmed with functional programming languages. For the really crusty, it can be lisp. Or scheme. Or Haskell. Recently, I like the look of clean. Whatever. But please, not more C, or C++, or java brain damage. Not even Python.
Ok, I’m ready for objections from the whining Asperger’s sufferers who will undoubtedly assure me that a “really useful engine” (attributed to one Thomas the Tank, I believe) can’t work that way.
Musings of a crusty old fart 
Dont quite know what you are writing about in regard to not even python. There was a reason perl was successful time ago, they had less capable computers too š
For a few reference points on the design space, refer to Taming the IXP Network Processor, Programming the Greedy Cam Machine, and maybe a graph reducer like the Reduceron.
Supposedly, graph reducers failed commercially not because they were a bad idea, and not because register or stack architectures were better, but because by the time an experimental processor gets to market, the conventional processors were already several generations ahead in clock speed. This is no longer an issue; clock speeds have been stagnant for years, while memory latencies dominate and power consumption is completely out of control.
In this sort of context, someone always brings up the parallelism implicit in a program’s structure (not to be confused with data parallelism). Indeed, functional languages do make this control parallelism clear, and even measurable. It has been measured, and it turns out not to be worthwhile. (There is at least one paper, but I’ve forgotten what it was titled.)
As an aside, you can’t escape state; all programming formalisms have it. In the lambda calculus, state is introduced by lambda bindings. What you want to escape is mutation of data; the so-called separation of Church and state. Nor can you rule out side effects. As Simon Peyton-Jones likes to point out, all you can do without side effects is push a button and watch the box get hot for a while.
In conclusion, there is no conclusion, except that you’re not the only one dissatisfied with the status quo. And don’t feel bad if your head explodes while trying to understand the Greedy CAM; you’re not alone there, either.
An insightful article. Indeed, it is time that programming moves forward.
You say “The only way to know the result of a program is to run the program”, but that’s a recognised limitation all the way up the theory (and into the inner depths of philosophy), rather than a limitation of a limited architecture. By definition, “running the program” is how one finds the result of any program. Even if the compiler is doing 90% of the work, it is still running the program (albeit without a complete set of input). Functional languages impose rules which mean that a program will always give the same result, so you only have to run it once – but once is still more often than not at all. Was that closer to what you meant?
Because the gratuitous insult-flinging only serves to highlight arrogance, ignorance, and weakness of character. It just looks even worse when it’s uttered in the same breath as a statement that’s *obviously* Really Fucking Stupid.
While i’m not sure yet if i agree or disagree, this is the most thought provoking thing i’ve read in a while. Cheers š
Cool! It’s interesting the way we start to question Von Newmann’s model when studying and understanding some basics of the functional programming paradigm(I have been studying some Haskell’s basics for 2 months) . Maybe it’s a natural process š
I believe that soon we will need a paradigm shift on the current state-oriented computer. The industry’s consciousness is changing.. The monolithic model that consisted on “heaps of power or torque” is already replaced by multiple cores running at slower clocks with emphasis on low power consumption and efficiency.
Well, i’m curious to know what will happen in the future! That’s a cool architecture of yours and i bet that can make any lambda-person horny!
so this is as opposed to bipolar lisp developers?
Don’t forget FPGAs. You can plug one into an Opteron socket.
The Sun T2 has 4 on-die memory controllers. It supports 64 threads and does hardly any speculation. The instruction set is SPARC v9.
Clean is a good language. Try Epigram, which feels like an extension of pattern matching.
Thankyou! — this is the zen “stick whack” I needed to move forward.
I get! I will forever be in your debt.
the only way to know the result of the program is to run the program? well, obviously. what else do you propose? some magical P=NP device which would send back the answer from the future?
since your architecture has state (since it has memory), you still need to coordinate processes that happen on that state. you need higher level coordination. a serial instruction stream (high-level instructions, yes, but still serial) which specifies what happens -before- something else.
i agree procedural languages are terrible in that they discard all parallelism, and then current CPU designs waste so many resources trying to suck that parallelism back out from the instruction stream. but the way forward is to have better languages and compilers which don’t discard this parallelism, and to have the localization of state, so that orderings only have to be enforced locally, rather than across the whole program.
i would like to see reconfigurable logic make its way into computers (especially for crypto), perhaps even with dma for an independent process. but then you’ve still got the coordination problem.
(yes, i come from a hardware background, but i’ve been doing python and scala for a while now)
Agreed. Too much time and effort has been spent on Von Neumann designs and no one (except for the really academic seen-the-light kind of Haskell/Lisp guys) has really expressed any interest in investigating different ways we could’ve done things.
Companies like Intel and AMD just keep pushing out their latest and greatest *same-old-crud*. We can’t keep increasing the speed of CPUs and the memory bus ad infinitum. It has to stop somewhere.
Write some code. Write a mockup on an FPGA of an 8 bit processor that works how you think it should.
Once you start adding things like cache, you’ll begin to see why some of what you rattle off in a few words is incredibly hard.
Time is always relavent. You can’t wave state away that easily. It might work in your head, but once you actually start writing some verilog, you’ll see how hard it is.
Memory is slow. Damned slow, and when you run out of memory, things get scary. If you have your data in a register, it’s like eating an apple on your desk. If it is in cache, it’s like walking to the fridge. If it is in memory, it is like driving to the store, buy some food, coming home and eating it. If it is on disk, it’s like going to farm and fleet, buying some seeds…….
Your plan will waste a lot of memory as well, which means more seeds to plant. Doing GC in hardware is scary.
Hardware is ugly for a reason. Even the things that appear clean at first are ugly underneath.