I’ve begun to establish a reasonably sound design pattern for the lexical analyzer. Of course this isn’t intended to be an ideal solution to the general case of writing a tokenizer for any language, it does not support any kind of short hand for describing token structure for example. But it isn’t overly complex and at this stage supports some of the common tokens seen in C, C++ or C#.
Monthly Archives: December 2013
A Superlean Inter-Thread Queue
Download the complete sample solution – scroll to bottom of post.
There are times when a design calls for an ability to pass information from one thread to another within an application. The Actor design pattern hinges upon such a capability as do other bespoke architectures in which dedicated threads play a central role. In an asynchronous design information passes between threads by queuing requests to a thread pool, the operating system internally schedules the processing of queued work by selecting some arbitrary thread within the thread pool and causing that thread to invoke a callback that you supply directly or indirectly like when using async/await.
Shakey The Robot
When I was starting out in my career I retained my interest in technology, electronics and so on and one day stumbled upon a truly fascinating book – a book I recommend to people even today because of its in-depth yet readable coverage of AI and robotics – the book is The Thinking Computer – Mind Inside Matter. This book is about AI and LISP and stuff one would expect but its very readable, not overly theoretical – it also has a fascinating chapter about a 1966 robot project setup at Stanford – the robot was named Shakey.
An Experimenter’s Robot Framework
When I was in my mid teens and still living in Liverpool I spent a lot of time reading technical books and magazines, electronics was my hobby but I hadn’t started my formal education in that field yet. One day I picked up a special Christmas issue of Electronics Today International (ETI) – the issue was entitled Electronics Tomorrow and it contained several fascinating articles that speculated on the future of electronics, with hints at predicting a perceived future.
Never say “never”
The increasing sophistication of modern processors, operating systems, programming languages and design patterns when combined with our unceasing determination to tackle increasingly ambitious problems has led to a greatly increased scope for dissatisfactory performance. Developers must be more vigilante than ever before if they want to satisfy their customer’s expectations with respect to speed, latency and costs.
Lexical Analysis With F# – Part 4
Confronting immutability
As I’ve been working on making the lexical analyzer (hereafter called “tokenizer”) more complete and simpler to understand, it’s begun to really dawn on me why immutability is so very important. I’ve begun to realize that by eliminating the traditional concept of assignment we are left only with function invocation.
Computational Art – Animated Music
I recently stumbled upon the craft of animating a musical score using software and musical expertise combined. Someone very active in this field is Stephen Malinowski who’s been experimenting and exploring this idea for some years.
Computational Art – Mandelbox
I recall once writing a small program on a Sinclair QL which drew an image of the Mandelbrot set on the screen, I was amazed at how a simple looking recursive function could possibly lead to such an incredibly complicated looking pattern.
Lexical Analysis With F# – Part 3
Although the initial version of a lexical analyzer represents the input file as a List, it also exposes tokens in a crude “one at a time” manner, burdening the token consumer with the responsibility of passing the previous token into the tokenizer function when requesting the next token.
Lexical Analysis With F# – Part 2
In the previous post I presented a simple lexical analyzer created purely as a means for me to get deeper into working with the F# language, such projects are a valuable means of subjecting oneself to realistic pressures so that one is compelled to deliver some kind of working solution.
