Monday, December 31, 2007

New year, new project

I mentioned earlier that I am a bit frustrated with SUnit as it currently is. This is because every time I go look at it or have to modify it slightly for something, I end up thinking that it is in need of refactoring.

Since this refactoring is of the large scale variety, modifying the current SUnit would possibly break existing test cases which work. Therefore, I decided SUnit needs to be rewritten.

And well, at least a mock draft of that has just begun. The name of this project was chosen because I think SUnit has potential for much more than just unit tests. I think the current naming convention (unit tests, tests cases, test results) does us a disservice because it is as if it implicitly prohibited any use of SUnit other than to write unit tests, which is silly.

What we do with tests, regardless of whether they are unit tests, or validations, or benchmark testing, are all instances of the same thing: assessments on various objects such as code, domain objects, or performance characteristics.

Therefore this new stuff is called Assessments.

We'll see how it goes.

Saturday, December 29, 2007

Optimizing C compilers, hmmm...

My first structured programming language was Pascal. Back in the day I became quite proficient at squeezing the size of the compiled executable by means of careful development. I also spent considerable time looking at what worked and what didn't in terms of execution speed. Further exposure to assembler only emphasized such learning.

For example, I still remember to this day that back then I got ahold of a library to manage fancy input fields called GICSTR. The original had about 4600 bytes of compiled code. By the time I was done with it, I had a rewritten one which required about 2200 bytes of compiled code. Both did exactly the same things.

Then came Smalltalk and for many years that knowledge stayed dormant, at least as far as things like CPU instruction scheduling was concerned. While I like optimization in Smalltalk as well, it's not the same thing as counting CPU cycles.

So nowadays I write C code, and I am somewhat disappointed to find out that even after all the talk about SSA, optimizing compilers and so on I can still go ahead, tweak C source code with assembler in mind, and lo and behold the VM runs faster. For example, it looks like I may have just squeezed another 5-10% out of the nfib benchmark by further massaging of the small integer addition/subtraction primitives.

But besides the speedups which are always nice, what bothers me is that I should not have to care about the manipulations I am doing. What is this optimizing compiler business if I still have to do manual work? Sigh...

In the mean time, I guess I will end up swapping the old Pascal/assembler optimization library in so I can modernize it with more current C and x86 assumptions. More speed is on the way.

Thursday, December 27, 2007

Revolution, evolution, and convolution

Many times I've heard, in the context of software development, that a computer revolution is still needed, and how this somehow means we need to find out what is going to be the next technological development that will change everything.

The way I see it, what we are looking for when we ask for the next set of shiny, glass programming language beads is an excuse to start over from scratch and to hide our inability to deal with our current obligations using current technology under the rug.

Back in the day, Java was the biggest thing. A-ha. Today, it's Ruby or Python or whatever. And tomorrow there shall be many others as well. At the same time, we still use C and we do not even dare say it's old news.

More importantly, however: have we satisfactorily addressed the underlying issues of software engineering, or do we still do the following?

  • Software module dependency is handled poorly, e.g.: Vista.
  • Buggy software is shipped regardless, e.g.: Leopard.
  • A high percentage of software projects still fails miserably.
  • Projects deemed successful do not do what the customers wanted in the first place.
  • And, more than ever, it is possible to call oneself a developer without even becoming familiar with things like modular arithmetic.
So if this is the state of affairs, how can we claim to be prepared to make sound judgments about our past, current and future technology?

I still do not find proper reasons to justify our apparent need to start over, or to invent the future if you will, because I think we cannot do our job properly today. What kind of next big thing can we invent if we have not even mastered what we already have? How can we claim the new stuff will solve our problems if we have not yet clearly understood what those problems are?

Therefore, my friend, when I hear we need a new approach to everything and therefore all our problems will magically go away, my answer is no. Domain problems will remain just as hard regardless of whether we use Smalltalk, assembler or Cobol to address them. That is the point here: we are getting confused because we think the domain issue is a technology issue.

But why do we do that? Because usually domain problems can be solved with rudimentary technological tools. We are not afraid of the hammer, of course not. What we are afraid of is building a house using a hammer. Since that is scary then we claim we need the next big thing, perhaps an extreme hammer, to build houses properly.

Of course we use the extreme hammer with its bright red or military camouflage handle as some sort of talisman so we can fool ourselves into feeling confident enough to go do what we have to, but we do not say that.

So if we need to build a house, do we study civil engineering or architecture? Of course not, because we are scared of that too.

More than a revolution, what we need is a good dose of evolution to go past our current state of convolution.

Wednesday, December 26, 2007

Nothing to do, so...

So I thought I'd go back to one of my lovely past times for a while, and found a couple things I didn't know. How about this?

The sum of the binomial coefficients (n, j), for 0 <= j <= k, is the same as the sum of the binomial coefficients (n-j-1, k-j) * 2^j, for 0 <= j <= k.

Pick any place inside Pascal's triangle, and the consider the diagonals going up from the place you selected. The sum of all the numbers in both diagonals is a power of two.

Lovely stuff!

More hash book for you

I'm getting good feedback, and this results in more work and more text: the draft just reached 460 pages.

Tuesday, December 25, 2007

Time

As of late, I've been spending considerable time thinking about the passage of time. The writing is on the wall: there's just too little of it available.

Sometimes when I say things like that I get comments such as "ah but you are young, what do you care about running out of time". But you know what? I want to write books today because they require an amount of energy and dedication that I seem to be capable of spending today, and I do not know if I will have that capability in the future.

Just as my eyes progressively lose their ability to focus on near objects, I feel the effect of the years in other things too. For example, I've noticed that the amount of attention I put into everyday stuff has diminished considerably over time. A friend of mine commented on something that I am becoming strangely familiar with as well: a particular kind of context switch becomes more costly, and stack spills in short term memory cause what appears to be a freeze of the task scheduler. Is that a sign of getting bored and just not paying as much attention to everyday stuff anymore? Is that a sign of being tired? Or is it an early sign of things to come?

In any case, one only has to look at older people to know what's coming. Therefore, one thing is for sure: I am not going to wait until it's too late. The time to do things is now.

Hash book...

I really need to stop working on it, because everytime I touch it what happens is that it grows. Now it is 458 pages, has a new introduction section of 4 pages, and the final chapter now has a bonus exercise section with 7 exercises... including a truly beautiful one regarding binomial coefficients. Sigh... at least, it's RC2 now.

Together with the mentoring course book, I am up to 796 pages --- and since I have 15 or so pages already written for the AGC book, I now realize that I am past the 800 page milestone. Amazing.

And watch out, because this Saturday my sister and I will create the cover for the mentoring book. Ah my friend, I am telling you, it's a work of art.

Monday, December 24, 2007

Happy holidays!

Ah my friend, I've been working on a few little gifts for you in the last few days... fine, I admit they were pet peeves of mine to begin with. But if everything goes through, then at some point in the future you will have:

  • 12% faster nfib benchmark,
  • up to 40% faster message lookup in some cases,
  • faster small integer addition and subtraction,
  • faster small integer multiplication, and
  • >2.5x faster largeInteger bitAnd: smallInteger when both are positive.
Cheers!

PS: Hmmm... I had further thoughts about method lookup and I think it can be made to be significantly faster than what is stated above. We shall see.

Saturday, December 22, 2007

Work, work, work and more work

I read somewhere that when you take care of your job and meet your responsibilities, the reward is more work and more responsibilities. Sigh, so true.

So what does the pending task list look like now? Let's see...

  • I am growing tired of the current implementation of SUnit. Each time I extend it or talk about its extensions it turns out there's more stuff to refactor because it gets in the way. Test selector lookup is quite inefficient by design, creating TestCase instances and filling the test selector in is certainly hackish, making subclasses to get specialized behavior is difficult because code is not refactored, etc. I am almost certain it needs to be rewritten from scratch. One of these days, that is exactly what will happen.
  • I need to write a little introduction to the hash book, thus making it RC2.
  • Further work needs to be done regarding the book covers. The one for the hash book is close to getting done, the one for the mentoring course looks good on paper and now it needs to be flushed out for real.
The amount of work required to do anything of decent quality is just amazing. I feel the need to offload work that is almost done just so that I can just swap out all that stuff and swap in other things that I need to take care of. Hopefully the books will be (really) done soon.

Thursday, December 20, 2007

Material from Smalltalks 2007

Ah, my friend, here are some links with material from Smalltalks 2007.

The presentation materials are here.

There are two sets of photos available: this one (taken by me and my sister) and this one (taken by James Foster). There is also a set of photos of Buenos Aires (taken by James) here.

Finally, the video* shown during the conference breaks is here. It includes greetings from many people (thank you!), as well as a number of photos from some Smalltalk projects and research labs that are doing very well.

Enjoy!

PS: you can get the video in other formats here.

*: the video is encoded as an AVI file with DivX for the video and LAME MP3 CBR for the audio, with the usual one second audio preload. Its duration is almost 14 minutes.

Sunday, December 16, 2007

Photos from James Foster

James Foster of GemStone fame provides two sets of photos taken during the time he spent in Buenos Aires for the Smalltalks 2007 conference. Enjoy!

Photos of Buenos Aires.

Photos of Smalltalks 2007.

We're still working on the official media package of the conference. Stay tuned for more stuff!

Friday, December 14, 2007

Smalltalks 2007, thanks

Conferences like Smalltalks 2007 do not happen because of a few people. They happen because of everybody. Therefore, I'd like to take some time to express my gratitude.

I'd like to thank the people that took time to come over and join us at the conference. Without them, our meeting would have been meaningless. Thank you for your enthusiasm, your efforts, your collaboration, your energies and your participation, as well as your numerous messages with nothing but kind words for us the organizers.

Then there were the volunteers who helped us with the organization of the conference. There was just way too much stuff to do besides hosting the event, such as the preparation of the bags with goodies for everyone, or manning the registration desk to assist people coming in.

In addition there were the sponsors of the event. Thanks to people from ESUG, Cincom, Instantiations, GemStone, Snoop Consulting, UBA, LIFIA, Mercap, Caesar Systems, InfOil and Seminarios Athena we had everything from the large room with movie screen for the conference to coffee/tea and snacks for the attendants. Thank you so much to every one of you for your support. It really made a huge difference.

And then, well... after all is said and done, it was four of us: Andres Fortier, Hernan Wilkinson, Leandro Caniglia and myself. Thank you for this wonderful experience of organizing a conference with full confidence and trust in each other. Without this kind of collaborative effort, Smalltalks 2007 would simply not have happened.

Cheers to everyone for this success!

Smalltalks 2007, Wednesday

Finally, we had a GLASS workshop on Wednesday. About 20 computers were used to develop a website with very little effort, which was hosted on James Foster's laptop.

So let's see... you set up your repository, and then creating 2-3 classes and writing these most silly of methods gives you a website with photos, CSS, a login window, and content which is rendered on the fly?

I am telling you: editing HTML by hand or even by tools looks silly by comparison. Why bother with that when you can simply use a code browser in your web browser and do as you royally please, knowing that everything you are doing is in this rock solid persistent back end called GemStone? Why bother with anything else? GLASS will make a whole bunch of stuff appear to be obsolete, and hey... I am considering using it myself to replace my flaky FTP/HTTP servers. I want nothing less, and nothing else.

Smalltalks 2007, Tuesday

Tuesday began with a presentation on software development which pointed out several things that cause software projects to fail, such as the vicious circle of buggy code maintained by experts who hack the system further in response to pressure from customers. In addition, to prevent these problems, a number of suggestions were made. For example, there should be a strong culture of testing, and there should be a separate QA team with minimal interaction with developers so that there is no conflict of interest. This lead to more than 80% code coverage, extremely low QA test recidivism, and a logarithmic increase of code size which in turn can be seen as code refactoring working its wonders.

Then there was a presentation about a complex software system that runs both on a desktop and a PDA. It is used to track the activities of a company that recycles clothing and other materials. So for example, the problems that are addressed by the software are the routing of drivers to more than 1200 containers, their activities, the tracking of how much stuff was picked up at each location, and many other logistic issues.

Since the next presentation was mine, I am not going to comment much on it.

After that, the Argentine army came over and talked about their experience with developing applications in Smalltalk. Here are some of the most interesting points.

  • Smalltalk is very easy to learn.
  • There is little ramp up time for new developers. If they only know OOP, then they can contribute within 80 hours. If they know Smalltalk as well, then they are productive within 40 hours.
  • Adapting the software to new requirements is easy and can be accomplished without going through major trouble.
What is interesting about these is that if you ask any Smalltalk developer, you will hear the same things. But in this case, there is a study backing up these conclusions against the experience observed in other programming languages. In other words, it is not as biased as a Smalltalk developer would be, and as such it can be seen as external confirmation of what we all feel to be true.

During this presentation, a number of Smalltalk simulation applications were shown. All of them included rich graphical presentation, including for example the overlay of information of military value on top of actual satellite photos.

Then, just before lunch, we saw a presentation on context sensitive applications and physical hypermedia. I wish I had been able to take more pictures, but there was so much information that I just could not make time to use the camera. The first part was about a number of software models that can be used to deal with issues that come up in context sensitive applications (in which software is aware of what the user is doing besides what the user is actively telling the application), and the second part included a demo of a physical hypermedia application running on a PDA which allows users to navigate a city in two completely different ways: by following internet-like links to get more information, and by following physical links which are interpreted in a different way such as e.g.: the user wants to go somewhere, but the traversal is done by the user not the computer, and as such different rules apply. I am really not doing any justice to the presentation by compressing it down to a paragraph, but if I had to file it out here it would take me 20 pages or so.

After lunch, I attended a block of 3 presentations out of a possible 6. First, there was a presentation on how to integrate the SCRUM process to VW's Refactoring Browser in a multi user fashion. It included the following nicest of features: once you tell the tool which item you are working on, then it collects all the changes into a changeset. Then, when you are done, it sprinkles the changeset into the corresponding packages, does all the merge work for you, and even publishes it with the proper version numbers. Everything that can be done automatically is performed for you.

Then, there was a presentation about how to interact with objects in a graphical manner. Mostly, the conclusion is that one should not use a UI builder to build UIs because it causes all sorts of problems and irritations. Instead, if one could attach a set of halos to an open, active window, then one could simply switch the whole thing into edit mode, display anchors and other visual indications to show how widgets are layed out, and then simply change them on the real thing. It was followed by a demo of the actual thing, which was quite impressive.

After that there was a presentation about Squeak 3.10, which included many talking points about the development processes that are guiding the latest release of Squeak. We also saw, besides the release image, an image with a number of things that did not work in previous releases. Hopefully these will see the light of day soon.

And thus we went into the final presentation block of the conference. First we saw a presentation on GLASS given by James Foster of GemStone, including a demo of the GLASS appliance being actively developed right now. What could I say about this other than it was very impressive? The default configuration you get will allow web sites to serve 15 pages per second with a 10 minute timeout in under 4 gigabytes --- for free!

Immediately after that we had a presentation about Morphic 3.0, which includes a UI written independently of the notion of square pixels. In fact, besides having something like 10x less code than the current Morphic 2.0 implementation, it can do arbitrary coordinate transformations on graphics. So therefore if you have a rectangular map, you can project it into the screen via the cylindrical transformation of choice and presto, there you see a map on the screen. Or you could also apply a spherical transformation and then you see an Earth. It also includes other goodies such as subpixel rendering.

And finally, we had a presentation on controlling robots from Squeak via Bluetooth with the intention to play soccer with them. This is not easy to do at all, because you have to interpret images you receive from an assortment of cameras to figure out what to do quickly enough so that your robots can operate as a team. While showing live soccer would have been impossible due to the fact that being able to play currently requires something like 2 days' worth of calibration efforts, we did see the the simulators running on the screen. Amazing.

So this is how the conference ended on Tuesday.

Smalltalks 2007, Monday

Maximo Prieto gave a very nice keynote. In a way, it hurts to see that we have not done much since Simula-67, some 40 years ago. What are we doing today? Objects, classes, messages, single inheritance, polymorphism. All of this existed decades ago. Where's the new stuff? Is there new stuff to begin with or not? Is there, as is there in music, a limit that has been reached past which it is nearly impossible to innovate? Thus the keynote's more than appropriate title, "And now what?".

The next presentation was on how to structure a large financial system as a collection of subsystems which are late bound via the specification of the system composite. This allows catering to different customers with a single code base because what they receive is a different specification of what their software looks like.

The presentation after that was about a large scale refactoring of an oil and gas industry system so that there is a distinction between services provided by the developers so that users can write their own software, and what users actually do to model their business processes.

Before lunch, there was also a presentation on several tools to assist the integration of code changes in large software projects. The big feature of the presentation was a live demo of an Envy-based automatic code integrator, complete with publishing capabilities pre/post condition checking.

During the first afternoon block, I attended two presentations. The first one was Feature Driving Browser, which included the following interesting observation. If you have a lot of tests and a lot of code, sooner or later you will have the problem of tracking down which piece of code is causing some test to fail. If there is a lot of code, then this can be time consuming. What the tool could do for you is to compare the code coverage of tests that pass to that of the tests that fail. The code that is run exclusively by the tests that fail can now be considered as prime suspects.

Then, there was a presentation about how to implement true method delegation (a la Self) in VisualWorks. It was very interesting to realize that delegation is equivalent to a super message send with the exception that the method lookup can begin in any arbitrary class. While the implementation dealt with self/super references properly, some problems such as direct references to instance variables remain.

Then, back into the large room, I attended a very nice introduction to Seaside. While I could not pay a lot of attention to the next one regarding what (to me) appeared to be a macro refactoring of Seaside to eliminate tons of repeated code, the day ended with ST + RIA in which a Flash UI was mounted on top of a traffic simulator written in Smalltalk. Complete with traffic lights, honking and different kinds of vehicles, while the Smalltalk server ran the simulation, the browser displayed the progress in a really nice looking manner.

Then there was dinner, of course. And in this way, Monday came to an end.

Smalltalks 2007 recap

Ok... so after coming back from being very tired, and being awake again, it is time for the organizers to do the rest of the homework.

First of all, here are some stats from the conference. We had almost 300 registrations, and out of those about 200 people attended the conference (perhaps a few more that self registered at the registration desk). One could think this is bad, but it is quite the contrary. See, in Argentina there isn't a strong culture of companies giving employees days off so they can go to conferences. As such, some people could only come to one or maybe two of the conference days. In other cases, we heard of things such as group of 6 developers who wanted to go to the whole conference, but that due to production issues it ended being the case that one developer could go to the GLASS workshop and that was it. It is our hope that in the future this will change and that things will be easier for everybody in Smalltalks 2008.

Oh, by the way... it is pretty much obvious that there will be a Smalltalks 2008, 2009 and so on.

We are currently working on the photos, the materials, the slides, and everything else. These will be released in the next few days. In the mean time, I will be writing short notes about the presentations I attended.

More to come...

Wednesday, December 12, 2007

Conference so far...

I am sitting at the GemStone workshop, which will start in about 10 minutes. I have not had much of a chance to write about the conference, so let's see what I can do in little time.

First of all, it has been a great experience to be an organizer of a conference. And, in addition to that, the conference presentations have been extremely interesting and very well prepared, and public reaction has been very positive. It may be too early to tell right now --- after all, Smalltalks 2007 has not ended just yet --- but Smalltalks 2008 has already been mentioned by this year's attendants.

In the next few days, after we have a chance to wind down, we will make the videos and other material of the conference available to others.

Well... time to go. More later.

Monday, December 10, 2007

Smalltalks 2007, here we go

Today is the day.

I woke up to the sound of thunderstorms. The first cell went by half an hour ago, and the second cell is now dropping water and light flashes around here. Hopefully they will go quieter in about an hour when I have to leave. Even then, it's just an issue of convenience more than anything else.

Registration count so far: 293.

Wednesday, December 05, 2007

Smalltalks 2007 --- Dinner on Monday

We would like to remind you that there is a dinner event planned for Monday, December 10, in a traditional barbeque restaurant near the conference site. The goal is to have a base price with access to a fixed menu (most likely consisting of red meat), with drinks and beverages to be paid for on a per person basis.  Although the details are not finalized at this time, we believe that the proximity of these places to Ciudad Universitaria compensates the eventual cost that may be associated.

If you are interested, please add your names here.

The reservation will be made on Saturday. It may be the case that more room is available for people without reservations, but this is not guaranteed.

See you there!

Monday, December 03, 2007

df -k Smalltalks2007

The registration count is now 262...

... cross that out: 265.

... another update: 269.

... latest count: 272.

... yet more people: 275.

... correction: 277.

... last check before I leave: 282.

... at Houston airport: 286.

... at Buenos Aires: 291.

Saturday, December 01, 2007

Hash book review is complete

I just finished going through the hash book once more. I fixed a few typos, a lot of paraphrasing, and a handful of instances of bad code. Thus I have reached the first Release Candidate PDF file, which is now 450 pages long.

Now, the mentoring course book needs to go through the same process. Currently, the draft is 324 pages long, with together with the hash book's 450 pages, brings the total pages ready to be published to 774.

More to come...

Side thought about OLPC

Every so often I hear politics oriented comments about the OLPC and things.

I think OLPC is great, because among the millions of kids that will get one in their hands, I hope we find a Gauss, a Ramanujan, or an Einstein. Giving them an OLPC is far better than what our education standards will do for them, because e.g.: the more time you spend in US public schools, the lower your IQ becomes.

If only Gauss and so many others had had a computer to write software in...

Tuesday, November 27, 2007

Smalltalks 2007 keeps growing

The Smalltalks 2007 conference just reached 230 registrations. Perhaps we will reach 256 :)...

Update 1: a few hours later, the count is 237...

Update 2: we just reached 240 registrations...

Update 3: and now the count is 247 registrations...

Update 4: so today, Thursday 11/29, at 5:35am PST, the Smalltalks 2007 conference reached 250 registrations.

Update 5: on Friday 11/30, at 5:00am PST, the Smalltalks 2007 conference reached 256 registrations.

Monday, November 26, 2007

Contrast

I received the first proof copy of the hash book today. It's currently sitting a couple feet away from me. I am happy because the geometry fixes worked perfectly. LaTeX is the best thing ever, the text looks beautiful and gorgeous. And after I checked on these things, I lost interest in looking at it for the time being... perhaps because I have a much better version that I am still reviewing, with lots of improvements and fixes.

To sort of celebrate the existence of the book, and yet without being too convinced that I should feel extraordinarily happy or anything like that, I went to get something to eat. You could say I used the book as an excuse to get nice food, but oh well.

I was sitting there, thinking about a million things as usual... then a couple arrived, sat next to me, started drinking, and then began talking.

So the lady works by arranging car loans, and today she dealt with $300K worth of those. She said she was still wired from work, as always, and so she had to distract herself somehow. After the comparatively small bottle of sake, she kept insisting the food of an average place was the best in the world.

The more she drank, the louder she got. Ironically, one of the restaurant employees asked her if she drank wine when she got home to be able to relax, and she said no. The bottle of sake laid empty, merely inches away from her.

The husband seemed unable to get a decent conversation going with anybody. His sentences had less than 5 words each. The wife then commented that she sacrificed her big truck to go eat there, that gas was so expensive, and overall... well, you know... they went on and on with the conversation you hear from people who are obviously drunk. Overreaching assertions, the need to approve or disapprove of everything in very strong terms regardless of its actual importance, an increase in narcissism, and so on. Then, when I got back home, I saw this video.

You know, what would have happened if I had usually engaged in any of the activities described in the previous paragraphs? What if I had the compulsive need to be distracted or entertained? What if I drank as a way to calm myself down? And what if, on top of all that, I had to deal with the hangover that comes after getting intoxicated? Among other things, I wouldn't have the first proof copy of the hash book next to me.

Like white and black, like night and day, like the difference between noise and music, chatter and poetry, scribbling and painting. Simple, pure, stark, clear, striking, beautiful contrast.

Sunday, November 25, 2007

More hash book polishing

I just got past a rough patch in the hash book. On page 251 of the draft, I found another instance of broken code. In this case, this caused two more pages to be added to the book, and as such the hash book is now 444 pages long. Hopefully I will get close to finishing the review soon.

No more this particular Starbucks for me

I used to come to this Starbucks because it's close to home and they have convenient plugs on the wall so I could stay and use their wifi service. But then, one day a couple months ago, they moved the large desk next to another set of plugs. I thought it seemed real odd... why bother moving stuff like that for no apparent reason... and then I became suspicious.

Of course, the new plugs next to the desk do not work.

I thought I'd give them the benefit of the doubt, so I brought it up a number of times. Every time, the answer was stuff like "we have not heard any complaints", "we do not know", "they should work", "we will bring it up with our manager", etc. Some of the employees know me by name, so it's not like the conversation is started from scratch each time.

So I stopped coming as often and finally I thought I'd do some judgment day kind of thing and came back to this particular Starbucks. Lo and behold, the plugs do not work.

That does it. I am tired of being lied to, regardless of whether it is intentional or by omission. Although I am on a different branch of business, I have been doing customer service as well. I have higher standards than this stuff.

Oh well... there are a million other places I can go, including other coffee shops (Starbucks or not) that treat customers better.

Hash book draft update

I decided to read the hash book from start to finish. As it's been a while since I finished it, I have forgotten enough of it so that reading it does not cause me to remember what I wrote.

The PDF file is 442 pages long. I started yesterday, and I reached page 151 a moment ago. I think I've touched almost every page. For the most part, I've rewritten sentences for clarity, or changed one word for another, or paraphrased some text so that the paragraph looks better, that kind of stuff. I also caught something like 3 typos. So far, so good.

And now, on page 151, I think I have found a mistake involving broken code. Sigh... oh well, better sooner than later. Hopefully I will be done reviewing it within the next few days.

Friday, November 23, 2007

Smalltalks 2007 update

The schedule for the Smalltalks 2007 conference now has a commentary written in English for each of the presentations. We just finished these, so it may take a little bit for all of them to become available, but we estimate they will be up within the next 24 hours. Enjoy!

Update: all content is now available.

Thursday, November 22, 2007

Ah, see? Now we begin to understand

Check this out:

"Narcissists mainly want to punish or defeat someone who has threatened their highly favorable views of themselves," the authors note. "People who are preoccupied with validating a grandiose self-image apparently find criticism highly upsetting and lash out against the source of it."

Here's the full article. Where have I seen such behavior before?...

The other day I smiled for some reason, and I had this ironic realization that if the story of what is going on in the universe were being written in a book, somewhere close to the latest text you would find something like:

And then, in an insignificant planet in the middle of nowhere, somebody that would live a minuscule amount of time like so many others of his kin, smiled.

The thought that came immediately after that was: "who cares!". Seriously, now... and we think things like our gold coins are worth anything because what?

Wednesday, November 21, 2007

Smalltalks 2007 growth

The registrations keep coming in. We're past 210 now...

Tuesday, November 20, 2007

Smalltalks 2007 @ 200

Following the release of the presentation schedule, there has been another surge of interest for the Smalltalks 2007 conference. As such, we have reached 200 registrations a moment ago.

This event is going to be something... in fact, I even got a comment stating that it is going to be the largest Smalltalk only conference in the Western hemisphere. I am not sure if it is or not, and quite frankly none of the organizers is into record breaking anyway, but on the other hand it is the perception of knowledgeable people that are not in the organizing team. So we will take it as some sort of external validation that says things are looking good. Thank you for your support!

So now, my friend... you didn't want to register for the conference because... ???... :)

OOOooooohhh...

What is this that I read? Good news for my job, perhaps?

Monday, November 19, 2007

Writing...

Last Friday a friend of mine asked how I had managed to have time to write:

  • A mentoring course on Smalltalk, ~350 pages...
  • And a hash book, ~450 pages...
The answer was simple. Those who have been reading my blog for a while will recognize it immediately.

I do not watch TV. I do not have many things.

And then the situation became clear for this guy.

After some time I thought well, perhaps by default people get the impression that writing is this huge expenditure of time and that it is impossible unless it is your job. But how many more people would write books if they saw how much within reach it really is? Here's my grain of sand in this regard.

For me, the writing process began in September of 2005. I was working 9-12 hour days at JP Morgan at the time, plus 1+ hour of commute each way. And even then, I managed to write most of the mentoring book while working on Kapital.

At first it was not easy. When I began, it was taking me about 2 hours to write a single page of what today I'd describe as gibberish. But through patience and perseverance, I got my writing skills sharpened again. Now, if I have everything thought out in advance, I can manage to produce 15 well polished pages in a handful of hours.

And with this practice and well honed technique, something very nice happened. I joined Cincom, and with that I got 8 hour days and zero commute because I work from home. In other words, all of a sudden, I got all this extra time to write I did not have before. And, to top it off, I was not nearly as exhausted anymore --- in part because of the shorter work day, but also courtesy of the training in hard work I got while at JP Morgan.

So let's see: no TV (I do not even have cable service), few distractions with things, no commute, I live on my own, I have all this writing and hard work training that is now paying off, and I have a passion for what I do.

It looks good so far, but there is a key ingredient missing for success: I have organized my life so that I can maximize the time I spend doing what I like doing best. The importance of this, and your ability to let go of things you used to care about, is critical.

To give you an idea, I do not have a schedule because every single day is a work day. I measure the few interruptions to work in events/month. I can tell you there is not a whole lot of them in the first place. For example, I hate the fact that I have not managed to get rid of accounting (two events/month of account/receipt reconciliation, paying bills, and getting taxes prepared in advance --- I am ready to file every January 1st), and laundry (about 2 events/month).

All of this, my life, has been engineered carefully so the consequence becomes unavoidable: with all this time and energy available, it is inevitable that books get written.

How many of us do not accomplish their goals in life because of distractions like TV? Isn't that sad?

And well, there is something else. I want to share the things I do with the people I care about. I find it extremely disappointing that my grandpa, who passed away a couple weeks ago, will not make it to see the Smalltalks 2007 conference. To his last moments, he was happy about how well it was going, about the books, and that I was going to be there soon. Just one more month on top of his 88 years and he would have seen the first proof copy of the hash book, complete with its beautiful first cover draft. But alas, it will not be.

If anything, the lesson I take from this is that if I am to honor his memory, the beautiful moments we spent together, and the things I learned from him, the last thing in the world that I can do is waste time on things that are not worth it. I do not want to ever look back and realize that I've missed on things like this because I was lazy, or because I didn't care enough, or because I knew I could but didn't. No my friend, not for me.

See? When you really want to, you can.

What is in your life?

Smalltalks 2007 schedule is out

So besides the 188 registrations, we will have this schedule.

You are coming, right?

Was this really Best Buy?

So I received this email entitled "We think this might help you decide to visit us again" from what appears to be genuinely Best Buy. What is the message body?

"Enter Text Here"

Ouch...

Sunday, November 18, 2007

What did I just do?

That is the question in my mind. Let's see...

I fixed a number of geometry issues with LaTeX for the hash and mentoring course books. I changed from draft compilation to non-draft compilation. I switched from openany to openodd for the documentclass. I made new PDF files for both books. Then, I spent a couple hours and made a book cover for the hash book. When I was satisfied, I saved a copy as a PDF. I created a Lulu project for the hash book from scratch, sent the files in, played with the settings some.

And then, I bought a copy of my own work.

I find it hard to believe. So, in a couple weeks or so, I will receive my first proof version of the hash book. It still has to be reviewed and things like that, but this experiment will tell me if there are further geometry issues, font issues, margin issues, legibility issues, or any other mess that needs to be addressed before the book goes into print.

If all goes well, the next one to receive such treatment will be the mentoring course.

These books are getting closer and closer to becoming a real physical thing that I will be able to put my hands on. And, well... after more than two years of writing, it feels really weird.

Conference schedule coming up

Tomorrow is the day when the conference schedule becomes public. Although we (the organizers) worked on it heavily and know it very well, it has happened to us that when we go back and take a look, we end up feeling impressed.

As of the latest count, we have received 182 registrations.

Saturday, November 17, 2007

AMD news, nonsensical related video

Check this out --- a Pentium MMX screenshot in an AMD news story. Erm... huh?

Thursday, November 15, 2007

Social events for Smalltalks 2007

We started a list of attendees for dinner on Sunday and Monday. Please add your name here so we can make the corresponding reservation.

Wednesday, November 14, 2007

Smalltalks 2007 update

The conference just went past 170 registrations. In addition, let me tell you... when the presentation schedule comes out next Monday, you will be gladly surprised...

Update: check this out. There's more where that came from...

Monday, November 12, 2007

Progress...

I just finished the first draft of the mentoring course book. It's 315 pages long, and together with the hash book's 433 pages the original work is now 748 pages long. The book on AGC still needs to be written...

Also, the Smalltalks 2007 conference reached 160 registrations today. We are roughly a month away, and it keeps growing. How fantastic!

Sunday, November 11, 2007

More stuff to do

There is literally so much to do, there is not enough time for it all.

So... to summarize, I was writing "The Art of Object Oriented Programming", a book of 7 chapters. Chapter 7 was on hashing, it grew too large, and now it is its own book at 433 pages. This leaves 338 pages in "The Art of Object Oriented Programming".

Now, chapter 6 is about something called "AGC is Garbage Collect", but I realize it's going to take about a year to write, and I also expect it will grow out of proportion. Therefore, I will split "The Art of Object Oriented Programming" once more. There will be "A Mentoring Course on Smalltalk", consisting of the remaining 338 pages except any reference to AGC. The good thing is that, after the split, the book will be done and ready to go.

This leaves the following action items.

  • Publish a series of math lecture notes by Leandro Caniglia... ETA a few months or so...
  • Publish the mentoring course and the hash book... ETA a few months or so...
  • Implement AGC and write the book about it... ETA 1 year or so...
  • Write the book about software development processes... ETA undefined at this point...
More to come.

PS: some days I really wish I had an assistant...

Saturday, November 10, 2007

Spam, or not spam...

So I just sent out an email from Gmail, and got this in return. Apparently, Windows Live Hotmail thinks Google generates spam. Not that Gmail has a very nice spam filter, no no...

Technical details of permanent failure:
PERM_FAILURE: SMTP Error (state 12): 550 SC-004 Mail rejected by Windows Live Hotmail for policy reasons. A block has been placed against your IP address because we have received complaints concerning mail coming from that IP address. If you are not an email/network admin please contact your E-mail/Internet Service Provider for help. Email/network admins, we recommend enrolling in our Junk E-Mail Reporting Program (JMRP), a free program intended to help senders remove unwanted recipients from their e-mail list: http://postmaster.live.com

Smalltalks 2007 extended to two days

Well well.... we just sent out an announcement regarding the fact that the Smalltalks 2007 conference in Buenos Aires will be extended to two days because:

  • 28 abstracts were submitted to the conference,
  • 150+ people registered so far.
This is wildly beyond anything we could have guessed before hand. As such, we have also moved from our previous room because its space for 100 became insufficient to hold the conference. Our new destination is the Aula Magna of Pabellon 1 with its sloped floor and seating capacity for 400.

I am telling you, the quality of the abstracts we received is something else. This is a conference you will not want to miss.

Saturday, November 03, 2007

The hash book

I just finished the hash book. The first complete draft is 433 pages long. The only things that remain to do are related to polishing.

Strange... I feel nothing special or out of the ordinary.

In any case, to get my bearings right with publishing, what I will do is to take a number of math lecture notes that I like a lot and publish a booklet with them. The experience with the small case will help ramping up the process to a whole book I am sure.

More to come...

Update: oh dear... it is beginning to sink in...

Friday, November 02, 2007

Smalltalks 2007 growing...

The Smalltalks 2007 conference to be held in Buenos Aires just went past 100 registrations, and we are still 5 weeks from the conference. We have also been getting high quality presentation proposals. I can't quite tell what they are yet, but it is going to be a very interesting gathering.

It's really exciting to see how large this is growing. It was even noted that it was likely that attendance could be larger than that of Smalltalk Solutions 2007!!!

If you would like to participate, all you need to do is to go over here and follow the registration procedure. It is very simple, basically all you need to do is to send an email to smalltalks2007 at gmail.

Don't miss out --- see you there!

Wednesday, October 31, 2007

Book growth

I went past the 750 page mark a while ago. The book drafts are now 338 (mentoring book) and 415 (hash book) pages long. The hash book is getting close... so close... I can feel the end is very near now.

In related news, I will most likely split the mentoring book again because chapter 6 on AGC will almost certainly balloon into its own book anyway. This would imply the following...

  • There will be three books.
  • Two of them might be published together soon.
  • The third one will likely take about a year to write.
And did I mention there are serious plans for a fourth book on team development processes?...

Golden standard

A grieving father won a nearly $11 million verdict Wednesday against a fundamentalist Kansas church that pickets military funerals in the belief that the war in Iraq is a punishment for the nation's tolerance of homosexuality.

Ok, so what are we going to do with the other guys that picket funerals of homosexuals with lovely signs stating that the deceased is going to hell and such?

Friday, October 26, 2007

Much better than TV

I have a neighbor spider. The first time I saw it, it was trying to live on the stairs casting its net from the handrail. Since then, it moved to the ceiling light in front of my street door. During the day, it squeezes into a ball and hides in the crannies of the sheet rock. At night, it goes down to its 7" net, and waits.

It is a beautiful, 1.5" long spider (including the legs). Its body is light brown in color, and its legs are tigeresque because they alternate between lighter and darker color stripes.

It has been growing, too. Apparently it's in a good place to hunt. It makes me happy because then my front door is free of pesky insects that I'd rather not put up with. I'd rather deal with the spider --- at least it doesn't go flying and buzzing around you all the time.

Just a moment ago I was about to go out and I saw it once more, slightly larger than last time, hanging upside down from its net, waiting. I observed it against the light, and I saw once more its legs and leg hairs. Beautiful and amazing creatures, spiders. I also observed its net, and again paid attention to how it becomes visible or invisible depending on the angle you look at it from.

So I was contemplating all of that wondering what pattern of perception allows the spider to do all this, and then came a big fly, one of those I find repulsive, and flew once around the front door area. I thought it was lucky, but I also considered that it might come back towards the light.

And sure enough, five or so seconds later, it came back and smacked right into the net of the spider, about 4 inches away from it. Then I got to see how spiders hunt for real.

Upon the fly hitting the net, the spider was on it within half a second. It collapsed into a ball around the fly, some tremors were observed, but within 10 seconds it was more than over. Spiders are no nonsense hunters.

Then came the interesting part. The spider took part of its own net and wrapped the fly in silk. It was amazing to see what appeared to be the spider getting tangled up in its own web, but no such thing. With anywhere between 4 and just 1 leg for support, the other legs made a very nice looking wrapped fly.

At this point, the spider's net had a major hole in it. But no worries: the spider went down a few inches to separate itself from the mess, and then just climbed back up towards the light. All of this of course with a leg or two carrying its dinner. When it got to the top of the broken net, it started cleaning itself up. All the legs were carefully scrubbed without losing track of food, and finally after 2 minutes of this, came dinner time. This was also very interesting, as the spider managed to just sit on the leftovers of its net with the fly next to it in such a way that it could eat without having to move or touch the fly any further with its legs.

They can show you all of this all they want on TV, but watching it with your own eyes is so much more satisfying.

PS: I found this photo that looks very much like it.

Thursday, October 25, 2007

Avoiding the mirror

Really? Exponential growth cannot go on forever? I could have never figured it out!

Check out the sobering stats --- we're about 30% in the red. Now of course, the actual percentage number does not matter. The issue here is that we are not in the black in the first place. In turn, this implies that sooner or later, this world of fancy, constant distraction and entertainment regardless of the cost is going to come to an end.

Given this knowledge, we can make a decision. Either we will care enough to modify our cancerous behavior in time, or a lot of people will die.

So, my friend: do we care? Apparently, not enough.

Monday, October 22, 2007

California fires

So the fires began due to downed power lines. Assuming the Santa Ana winds caused the power lines to go down, the number of fires started at the same time just begs the question...

... why are power lines that cannot take abuse from Santa Ana winds built in land that has a tendency to become tinder ready to go up in flames in a particularly uncontrollable fashion especially when Santa Ana winds are blowing?

So let's say we just figured this issue out because we didn't know any better. Fine. We are going to learn our lesson and do something about this so it does not happen again, right?

Right?

Saturday, October 20, 2007

df -k book*

Yesterday I broke the 700 page barrier. Right now, the book drafts stand at 338 and 366 pages. There is so much more to write...

Well, I better get back to it now. Hopefully in a bit I will finish the section on Number>>hash.

Update: some 8 hours later, the section is done. The hash book draft is now 377 pages.

Thursday, October 18, 2007

An answer to the wrong question

There has been a lot of talk recently about Giles Bowkett who asserted that the lack of a good debugger is a feature because it prevents detrimental developer behavior patterns.

So fine, let's take out the debugger. If you have no sharp tools around, certainly you can't injure yourself. But how are you going to cut a diamond without a diamond cutter? And how do you make buildings without a bulldozer? This is like saying that since cars cause car accidents, we should ban cars.

In my opinion, this is the answer to the wrong question for two reasons.

Back in the days, steering columns in cars were rigid, and thus if you hit something straight on the steering wheel would essentially attempt to move through your head with rather nasty results. So what happened? Nowadays, steering columns are telescopic and thus will collapse when such crashes occur. So, first reason: if the tool is unsafe, make it safe in the first place.

Now, what to do about crashes? Should we outlaw cars? Well, no. What we should do however is to make a better effort at driving responsibly. Here are some things we could try: do not cut in front of trucks, do not drive through railroad crossings when the barrier is low and the bells and lights are going, do not tailgate, do not pass on the shoulder, do not drive too fast for conditions, do not carelessly change the tires and suspension of your car, remember that the street is not Gran Turismo, and if you like driving vintage cars then install safety belts on them. Therefore, here is the second reason: we ought to stop blaming the tool for our own deficiencies.

I find it amazing that the discussion is not about how to stop developers from producing crappy code in the first place. No, not at all. Apparently, that can be like it is because applications written that way are fine and dandy, and therefore this problem is not seen even as an issue worth considering. But the mere existence of a debugger? Ahhh no, what an abomination of nature!

Sorry... this argument doesn't make sense to me.

What I would say is much more likely to be going on is that teaching people to be good developers is hard, and that since we shy away from that problem we choose to blame something that cannot defend itself to cover the fact that we are not addressing the real issue.

To put it more bluntly at the expense of restricting the context of applicability of the following argument: a nation in which giving people enough mastery of mathematics so that they are able to use them in real life problems is not deemed as being important or desirable will find it much harder to produce quality computer programmers, much less computer scientists or mathematicians. And why would that kind of education be a prerequisite to having responsible developers that in turn might think it is acceptable to rely on the debugger when their trained judgment tells them to?

Just my two cents here.

Wednesday, October 17, 2007

Señor Gnaritas

I found the following quote. Thank you, whoever you are!

"[...] simple is something [that] takes experience to truly appreciate. Simple never gets the respect it truly deserves until much pain has been experienced finding out why complex sucks."

Selected footage of talented sportsmen

Here's some footage I find very enjoyable. First, some selected clips of Romário de Souza Faria. There are some duplicates, but if you do not know who this guy is, then it's well worth watching.

Soccer 1. Soccer 2. Soccer 3. Soccer 4.

White hairs, playing past 40, whatever... all balls go in spectacularly when Romário is near by.

It's funny... at one point Romário said, essentially, "if you're going to call me to the national team, I will not come over to sit on the bench". So he got grounded and stopped being called... but then of course Brazil wasn't playing so well, and finally the coach gave in and called in the guy. For the game that they had to win to go to the world cup. Oh dear...

Well he put in a show, and what happened is in this clip (which, besides having commentary by Pelé, illustrates very well what it means to watch soccer in Brazil --- music and audio effects included).
See what happens when you put in, what, 20-30 years into something? Along the same lines, here is some amazing stuff by Semih Sayginer.

Billiards. And this: part 1, part 2 and part 3 (you may want to skip some of the ads).

Enjoy!

Sunday, October 14, 2007

Dolphin's Float>>hash

While working on the hash book, I've run into a discrepancy I can't reconcile. I was looking at Dolphin's number hashes and, to make testing easier, I reimplemented Dolphin's Float>>hash in VW (remember that in Dolphin, aFloat is the equivalent of VW's aDouble). When I tested the reimplementation, I couldn't get the values to match.

Unable to see what was going on by looking at the code, I went into a VW workspace and tried to calculate the hash of 3.25d according to my interpretation of Dolphin's implementation. The answer was 3. Dolphin's workspace, however, said 3.25 hash was 879642989.

Ok, so obviously there was a bug in what I wrote. I brought up a side by side arrangement of debuggers to see what was going on. Much to my astonishment, the calculations matched on both sides, and in both cases the answer was 3.

This just can't be. Could somebody please point at what I am doing wrong?

Update: aha --- compile time literals strike again! Recompiling the method in Dolphin produces the right answer now.

Friday, October 12, 2007

Status of the books

So, what's up in book land these days?

The mentoring book grew a bit because of a particularly nice exercise, and it is now 338 pages. No work is being done with it at this time, other than adding exercises when I find good ones. This lack of work on the mentoring book is not worrying to me because...

The hash book grew larger than the mentoring book last night, and it is now 341 pages. It's getting close and there is definitely the beginning of the buzz. Only 30 or so subsections of chapter 7 (of the hash book) plus small work here and there to go... it looks like a 400-500 page book to me.

Well, back to creating more stuff now.

Wednesday, October 10, 2007

Perfect, or nothing

For some reason I remembered some vague details about one time when Juan Manuel Fangio, an F1 driver, had this great performance smashing N lap records in consecutive laps.

Well, I found his own account of it. It is much more impressive than that. It is astonishing. Driving curves one gear higher than normal, flying in the air, driving through impossible places... to erase a 51 second deficit because of pit trouble, take the lead with one lap to go, and win a 5th World Championship (4th consecutive).

For some reason, reading his interview felt a bit too familiar... at times I couldn't help the sensation of knowing the feelings he was talking about all too well. Good stuff Juan Manuel, very good stuff. To your memory, my friend: cheers!

Tuesday, October 09, 2007

Smalltalks 2007 Announcement!

On December 10th, the First Smalltalk Conference of Argentina will take place in Buenos Aires. Interested in participating or perhaps doing a presentation? Send an email to smalltalks2007 at gmail dot com!

Here is the website for the conference. See you there!

Sunday, October 07, 2007

Regarding Interval>>hash

I was looking at different implementations of Interval>>hash, and when it became the time to measure I found that coming up with interesting cases was more difficult than I expected.

So... has anybody run into real situations in which a set of intervals or a dictionary with interval keys was useful? If so, can you share the nature of the intervals?

Thanks in advance!

Tuesday, October 02, 2007

Kapital in the news...

Check out these public glimpses at Kapital... it's nice that now I can use the term PCS! As impressive as it sounds, in my opinion the article does not do it enough justice :).

Enjoy!

Saturday, September 29, 2007

Native stack tuning in VW

The VisualWorks' VM uses a native stack to execute Smalltalk processes. When the space allocated to it runs out, however, it allocates stack frames inside the image as would happen in ST-80 (not to execute them --- rather, to make room in the native stack space). This is very inefficient because allocation is expensive, and also because then the VM will have to GC the allocated stack frames when they become unused.

You can tune the amount of memory the VM will allocate to the native stack at startup. This is done via a float multiplier stored at index 4 of this array:

ObjectMemory sizesAtStartup

The default value is 1.0, meaning 20kb. However, preliminary results obtained with SUnit Benchmarks indicate that in reality, for processes that have an average stack depth of between 20 and 50, it is much better to set this multiplier to the maximum amount of concurrent Smalltalk processes // 5.

In other words, if you have 1000 concurrent processes, you should set the index 4 of the array given above to 200.0, then do

ObjectMemory sizesAtStartup: modifiedArray

then save the image and restart it. The performance improvement factor can easily reach 10x. However, do not increase the multiplier way too much because it will slowly become counterproductive in the current VMs. As always, measuring is a good thing.

Enjoy!

Thursday, September 27, 2007

isKindOf: considered harmful

I recently made some changes to DLLCC that made evident the limitations of not using polymorphism. When loading a particular package, the process of doing so would spend about 7% of its execution in isKindOf:.

Although polymorphism eliminated that, I directed my attention to how isKindOf: itself was implemented. It turns out that it depended on includesBehavior:, and interestingly enough this method was doing essentially an isKindOf: check by sending notNil to superclass before delegating the work to it.

Thus, in the spirit of eliminating such things, I extended UndefinedObject to answer false when sent includesBehavior:, and then I just removed the notNil check in the implementation of Behavior.

This simple change results in a speedup factor of 1.5 in my test case, #() isKindOf: Object. Clearly, the JIT based PICs in the VM ate the notNil check for breakfast.

So my friend... what other low hanging performance improvement fruit is so much within our reach, waiting for us to recognize it as such?

Sunday, September 23, 2007

New hash functions mailing list

To avoid the spam in the newsgroup sci.crypt, and to have a proper place to talk about non cryptographic hash functions, we have decided to create a new hash function mailing list.

The URL is here: Hash Functions.

Enjoy!

Friday, September 21, 2007

And then there were two

I was writing one book called "The Art of Object Oriented Programming --- A mentoring course on Smalltalk". It had 7 chapters, and the draft had reached over 600 pages. But Chapter 7, entitled "Distinctions and hash", turned out to be something other than what I had planned.

Originally, I had budgeted 50 pages for the chapter. Before writing it, a more realistic estimate suggested 100 pages instead of 50. When it reached that mark, it was obvious that it would need another 100 pages to be complete. It is currently 223 pages of text, and it looks like it will require at least another 100 pages.

Note it is not even meant to describe everything there is to say about hash. Rather, its goal is to help make decisions about how to implement hashed collections and hash methods properly. Despite its limited scope, the sheer size and amount of exercises on paper so far imply that it is really much more than a chapter. This is particularly so because large areas that are worth exploring are still left unaddressed, even though they still fit with the scope of the presentation. Therefore, the chapter must be its own book.

So I just split the book into two. There will be the original book, which now stands at 334 pages with another 100-200 to go, and a new book called "Hashing in Smalltalk --- Theory and Practice". This new book is 284 pages already (text plus exercise solutions, no introduction, and the single chapter hasn't been broken into multiple chapters yet). No wonder it did not fit as a chapter in a book.

This also helps me address the issue I had with publishing. Volumes can only be 740 pages, and with half of chapter 6 still to be written, I was running out of space. This arrangement will make the material easier to manipulate.

There is so much more to go, still! But it looks like the hash book might get published before the mentoring course. We shall see.

Thursday, September 20, 2007

What drives you to create?

I see a problem, and I feel the urge to understand it, to find a beautiful pattern in which to encode its solution, and then to copy it into a computer without copying myself in the process. What drives me to do this? It feels nice, I guess.

But what drives you to create? Perhaps the same thing that drives this guy to make over 200 amazing drawings on an Etch-a-sketch. I wouldn't be able to do such things.

Meaningless as it is, on this planet in the middle of nowhere special... isn't what we crave to create worthy of our attention?

Talk at LASTUG last Monday

Well well, last Monday was a long day... it began with a lady's attempt to drive through the front end of my car without causing damage. She failed...

So anyway, the talk went reasonably well for being a beta. Although there are some aspects that need to be polished, there were no show stoppers or gross bugs in the content, so that makes me happy.

Something that was evident is that the idea of thinking about a good hash function causes a lot of curiosity! In this regard, the Hash Analysis Tool was nice because, besides automating all the tests and showing reasonably objective results, it allowed interaction by the audience. For instance, someone wanted to try out a hash function, so we simply implemented it in the Hash Analysis Tool framework in about one minute, and then we tried it against the built in data sets. This is something that I will try to take advantage of next time.

What, you haven't seen the Hash Analysis Tool yet? :)...

I'd also like to thank the people there for their patience with a beta presentation. Hopefully the next time it will not have those rough edges re: the sources of collisions.

Monday, September 17, 2007

Undeclared will not be emptied...

HolgerK made this comment in the IRC channel:

The mysteries of Undeclared: Why it's still not empty even though there are no traceable references

I found a workaround to that and similar issues. No need to print the result... just evaluate the expression below.

1000 factorial

Interesting, no? This seems to indicate that GC is perhaps being too conservative with regards to the native stack, thus catching references to stuff that should not be kept... but I have not found the issue yet. It may even be something else, but the hint appears to be plausible.

If anybody has seen something like this before, please let me know.

Thanks!

PS: I've seen this issue with large byte arrays, large integers, and byte strings. All "byte" objects... suspicious, suspicious...

Sunday, September 16, 2007

Book status

So, what has been going on lately with the book...

The draft reached 610 pages today. I am writing the last section of chapter 7, number 7.12. It has over 40 sub sections, and only a few of them have been written so far. A few of them promise to require lengthy treatment, such as the one regarding Double>>hash and Float>>hash.

My previous estimate of 200 pages for chapter 7 turned out to be wrong. Chapter 7, still missing significant amounts of material, is currently 221 pages. It looks like it will require at the very least another 100 pages of text, plus exercises. Sigh... maybe I should turn chapter 7 into its own book. I will have to think about that one.

Back to work!

Saturday, September 15, 2007

Ligget se

I saw once more that inject:into: was being used to add consecutive integers. But wouldn't it be nice if you could do this instead?

anInteger sumTo: someOtherInteger by: someSkippage

Note this is not a matter of providing a nice interface to a raw sum of the contents of intervals. This is much more. Let's begin with someSkippage = 1, in which the case above reduces to

anInteger sumTo: someOtherInteger

Of course, we define the sum to be zero if anInteger > someOtherInteger. Then, what do we do to implement Integer>>sumTo:? Something like inject:into:?

No my friend, we remember what Gauss did at age 10 when we has asked to sum 1 to 100. He quickly realized the answer was 50 * 101 = 5050, and throwing his slate on his desk pronounced "Ligget se", or "There it is".

Ok, so that works when anInteger = 1. But if it is 10, then the trick doesn't immediately work. Well we can make it work though. If we need to sum all integers between i and j, then we will sum i, i+1, ... , j. Ok, but that is the same as summing 1, 2,... , j-i+1, and then adding (i-1)(j-i+1) (i-1 times the amount of sum terms). We also see that when i=1, the "shift" term evaluates to zero as it should. The sum from 1 to j-i+1 is, by Gauss' technique, (j-i+1)(j-i+2) // 2, so the result evaluates quickly: (j-i+1)(j-i+2) // 2 + (i-1)(j-i+1). Very nice.

Let's try 13 sumTo: 29. Our formula says the answer should be 17 * 18 // 2 + (12 * 17) = 357. A quick inject:into: check verifies this is correct.

What if we want sumTo:by: now? Well, instead of adding i, i+1,... , j, we are going to add i, i+m, i+2m,... , j // m * m. How many terms are in this sum? Clearly, if i <= j (the case when the sum is not already known to be zero), we must have j - i // m + 1 terms (Smalltalk notation). Let's call h = j - i // m + 1 for convenience. Also, the largest term in the sum will be j' = j // m * m.

We can downshift this sum from i, i+m etc to 1, 2 etc just like we did in the previous case. First, we subtract i from all the terms (and then we just add back hi to the result). So now we have 0, m, 2m etc. Now we can divide all of those by m and then we have 0, 1, 2, etc (so we multiply by m after we calculate the inner sum).

Ok, so to get the result, first we sum 0 to j'//m, which is the same as the sum from 1 to j'//m. We know how to sum that because of Gauss, and it is just (j'//m)(j'//m + 1) // 2. Then we need to multiply all that mess by m, so we get m(j'//m)(j'//m + 1) // 2. And finally, we need to add hi to all of that. Then the whole expression is just m(j'//m)(j'//m + 1) // 2 + hi.

Let's try adding 3 to: 45 by: 8. Then, we have i = 3, j = 45, m = 8, h = 45 - 1 // 8 + 1 = 6 and j'//m = 45//8*8//8 = 5. Our formula says the sum must be 8*5*6 // 2 + (3 * 6) = 138. A quick inject:into: check verifies this is correct too.

Ligget se --- no inject:into: necessary.

No more SCO for you

SCO files for bankruptcy. Lovely 1, and Lovely 2. Apparently, it doesn't mean the lawsuits will go away though. Sigh...

Friday, September 14, 2007

LASTUG meeting next monday!

Next Monday, I will be giving a beta of a brand new presentation at LASTUG. It is entitled

Quality Measurements for Hash Functions

See you there!

Saturday, September 08, 2007

For those unfamiliar with Smalltalk's syntax

I found this very nice article by Wilf Lalonde at Roger Whitney's home page. It's entitled:

I can read C++ and Java but I can't read Smalltalk

Enjoy!

Wednesday, September 05, 2007

Hash Analysis Tool v2.2 available

I just published an improved version of the Hash Analysis Tool v2.0. Enjoy!

Update: also available, a new version of SUnit VM and SUnit Extensions with fixes and other enhancements.

Comment on multithreading

Who needs multithreaded applications in this context?

The problem we have is that today's computer programs do the same thing our computer programs accomplished when we had a P3/600. Think about it. Other than the FEW!!! applications that truly require huge amounts of CPU power such as video editing, what is it that we do today that we could not have done 20 years ago?

Certainly, if we could do it with 10 year old hardware, then we should be able to do it today. Unless of course our software is doing such huge amounts of unnecessary work we do not want to deal with, that we see the promise of multithreaded speedup as some sort of magical solution that will allow us to delay even further the critical need to spend brain cycles making our software significantly better.

Examples? Check this out: Java's HashMaps impose an average collision rate of at least ~5.84, and yet nobody complains about this. OOXML is an atrocity and everybody knows it. So my friend: do we need better multithreaded support for code, or do we have much lower hanging fruit to pick up first?

While we skip addressing the really difficult questions, chip manufacturers will have a much larger market for chips that cannot go faster than 4Ghz.

Saturday, September 01, 2007

Assorted thoughts

Sometime ago I used YouTube to find soccer videos and such. As I was watching, I realized that these players play professionally, say, no more than 30 years. And yet, their whole career is compacted into a video lasting a few minutes.

More or less the same happens with people that play billiards. Countless years of practice to be able to add, perhaps, a 30 second clip per year to your "this was my life" video.

How disproportionally unfair. And yet, there it is.

And then, if you think that is unfair because the work of most of the best among us can be squished down to something that at first sight looks insignificant (even though it becomes significant when you consider the amount of effort it takes, and then it becomes insignificant again when you consider our place in the universe), what is left for people whose lives are rather mundane and pretty much equal to that of millions of others?

What about the guy that plays clarinet on the street? What about us, just software developers writing stuff in a blog like oh so many others? What about the richest CEO in the world today?

The thing is, my friend, that we will follow the native population we helped exterminate years ago. When you visit their ruins, and you notice that some hundreds of years ago thousands of people lived in those places, then it becomes obvious. Their names, their lives, their family relationships, their work, their values, their laughter, their pain, their innermost insights on what is this thing about being alive... all of that, together with their riches, their possessions and their own ideas about wealth and ownership... all of that, gone into the fog of the past to never come back. In the best of cases, there will be some video of us, some of our random thoughts, and our names. But that is it: it will become our 15 minute, Andy Warhol claim to fame YouTube clip.

Today I went to a gas station to put gas in the car, and there was this teenager with fake diamond earrings selling fancy looking chocolates. He greeted me in terms of sir, and then he explained he was selling these things to win a trip to Magic Mountain. I told him I did not have the least idea of what Magic Mountain was. He explained that it is an amusement park with rollercoasters that go really high and so on. I asked him if what it boiled down to was that I had to give him money so he could go to an amusement park. His answer was that he had been working really hard all summer, selling these things, so he could have a chance to win this trip. I told him that I would spend money to go to an amusement park myself before I'd spend money for him to go. He went away in silence.

A couple weeks ago, I was going back home really late and as I was in the parking lot came this lady in her fifties asking out loud "am I that ugly?". I pretended to ignore the first question because I thought she was drunk, but she insisted. I turned around. She explained that she had finally accepted to go out with this guy that had wanted to date her for two weeks, and that he had been groping the servers, taking pictures of himself with them, and pretty much being rude and ignoring her. Thus her question, "am I that ugly?". As if not being attractive was the reasonable cause for his behavior. I took a look. Fancy flat sandal type shoes, short skirt, nice looking business like shirt, a short blazer, and reasonable appearance for someone her age. I could have answered "well my friend you are not 20 years old, so there is a limit to how beautiful you can be", but it was not the point to answer her question literally. So I told her something else. I said, "There are many women who look beautiful but are dreadful as people. I wouldn't want to touch, say, Lindsay Lohan with a 100 foot pole, no matter how beautiful she might be because I know that as soon as she opens her mouth to say something I will want to cry. The issue here seems to be that this guy did not want to be with you for who you were. All this dude cares about is how you look. Why would you want to be with someone that does not appreciate you as a person? He just gave back two, three or whatever hours of your life as a present to you. Why don't you go and do something you really enjoy doing with the time you now have?". There was this awkward silence for about five seconds. She stared at me, frozen. I stood still as well. Then she turned around, took a couple steps, and said "Yes, why do I want to be with someone who doesn't respect me? Yes, why? Right! I will go do something I want to do!". She started walking around in circles, insisting, "Yes! I will do whatever I want!". Then, she finally made up her mind, and strongly strolling down towards the street, the last I knew of her were her repeating words: "I will do what I want to do!".

Death is at the end of the road for all of us. What are you going to do before you reach it?

Tuesday, August 28, 2007

Hello, Cadence

Permutation work?... interesting!

Sunday, August 26, 2007

Vista comment overheard in the IRC channel

Check this out.

if you're playing music on vista, your network incoming [throughput] drops by 90%

Aha. So I want to spend on a new, ultra powerful computer to run Vista... why?

Saturday, August 25, 2007

Is this college mathematics?

Take a look: multiplication tables linked at a page mentioning courses such as "college" algebra.

You have to be so much kidding man. Multiplication tables with suggestions such as "you should memorize the gray area" and "remember 0 x anything = 0", with the official stamp of a college. College, you know? That place where people go after attending high school, which is where they go after attending grade school, which is where they learn how to multiply?

But there it is, it's reality. Shameful as this is, it's the mirror that shows us for who we are.

So... why do we even think we are prepared to discuss pretty much anything now? Seriously: what does it mean when people say "25% of 600,000 bridges carry more weight than they're designed to hold"?

You better be able to answer those questions properly whether you like mathematics or not. Or do you want to be dead now?

Gross confusion

So here I am, doing research. I go through Knuth's book, and in the particular section I am in he suggests hash(x) = ax, where the factor a has to be chosen properly so that it is within certain ranges and such that each of its bytes is not too close to any of the others or their complements.

Then he gives an example of a value of a, for a decimal computer in which the bytes can have values from 0 to 99. In "hundred"-decimal notation, it would be as shown below.

a = 61 25 42 33 71

Now, certainly, if we are working with a decimal computer, then yes... each byte is not too close to any of the others or their complements. But, in a binary computer, this looks quite differently. In hex, the value above is 33 bits long.

a = 1 6D 1A 8B 0B

So first, it doesn't even fit in 32 bits, and second, its bytes are not necessarily far away from each other or their complements. That is fine because the value of a was chosen for a decimal computer, not a binary one.

Want to cry? The value of a can be found in .h files via Google, even though it typically does not fit in 32 bit variables. Also, this choice of a has been roundly criticized for giving poor hash value distribution in binary computers. Well of course, it was never meant for a binary computer in the first place.

Sigh...

An update on the book

So... what does the to do list look like these days?

Chapter 6 on AGC is currently half written. Not much is being done on it at this time.

Chapter 7 regarding hash, on the other hand, is progressing nicely. I had originally estimated that Chapter 7 would be around 50 pages. Then I realized I was wrong and estimated 100 pages. It is currently 92 pages, and I would say it is no more than 50% written, probably less. Therefore, I have to increase the estimate for the chapter once more, to 200 pages. In addition, since I've run into problems, such as "there is no written information regarding this topic", I've had to research and write at the same time. Although it is a lot of fun, this slowed the pace of what would have otherwise been a simple file out. Oh well --- better get going!

Finally, the book gained an appendix. I hope the contents will be a very pleasant surprise.

The draft is currently 449 pages, and the pdf file no longer fits in a 3.5 inch floppy. There are about 1.1 computer megabytes' worth of LaTeX source code. There are no illustrations yet.

Time to write some more now.

Friday, August 24, 2007

A Pattern of Perception in action

Poor goalkeeper... there he goes, assuming as his strategy builds his objectives, that nothing will change. We live a fraction of a second in the past.

Wednesday, August 22, 2007

Fun with Intel x87 FSIN

For some arguments, asking an Intel x87 FPU to calculate the sin of a double precision floating point number, which has 53 bits in its mantissa, will result in a double precision floating point number with a mantissa that has its lower 40 bits set to zero. In other words, you get no more than 13 significant bits' worth of a result.

Check the answer to this expression for the very nice looking multiples of pi that cause this to happen:

(-1048576 to: 1048576) select: [:any | (((Double pi * any asDouble) sin mantissa: 53) bitAnd: 16rFFFFFFFFFF) = 0]

And a few fractions as well...

(-1048576 to: 1048576) select: [:any | (((Double pi / 1048576.0d * any asDouble) sin mantissa: 53) bitAnd: 16rFFFFFFFFFF) = 0]

(1 to: 1048576) select: [:any | (((Double pi / any asDouble) sin mantissa: 53) bitAnd: 16rFFFFFFFFFF) = 0]

I am very surprised, and I guess disappointed too, with this. I debugged the VM down to the FSIN instruction, and as far as I can see those are the results given by the Intel x87 FPU. The instruction FCOS seems to have the same behavior.

This does not look like it should be the case. Am I missing something here?

PS: If anybody runs these expressions in their image, would you mind letting me know the result and CPU type? Thanks!

Update: Indeed, it is a "bug" in x87 FPUs. Here is some proper explanation.

FPU questions

The new VM tests uncovered that double precision answers to trigonometric functions on SPARC cpus are not the same as those that x86 cpus give. A few least significant bits of difference wouldn't really matter that much, so I wrote a bit of code to allow checking for "almost equality" of floating point numbers.

When I went to see the tests that fail to see by how much "almost equality" should be adjusted by, however, for things such as double precision sin(pi) the difference is as large as 38 out of 53 bits of the mantissa. That is a huge delta! It feels really gross to fudge the precision of the test by a factor of 2^38.

But maybe there's something else at play here, perhaps I've been missing an obvious bit of information so far. I do not know what it is yet, but in the mean time hacking the tests to pass is definitely not right. Has anybody run into things like this before?

Incidentally, in the test above, the x86 FPU gives an answer closer to zero.

Friday, August 17, 2007

When 50% is much more than half

Check out this poll. Isn't it great to look in the mirror, even if what we see is not pretty? Also, check the graph below.


Update: the flier above came from PSE&G. They still owe me money from the last bill because they made an adjustment to my favor after I paid it. What did they do? Send me to collections, of course. Grrr!!!

Ok guys, prove me wrong

Perhaps this is a sign that I was not correct, and that we are indeed a serious country that takes care of their own people.

I doubt it. But I hope I am wrong this time.

Thursday, August 16, 2007

LASTUG next Monday!

Next Monday, I will be giving the presentation A Pattern of Perception at LASTUG in Pasadena. See you there!

Papers updated

There are new versions of two of the three Smalltalk papers I posted about recently. Go to the original post and get them!

Wednesday, August 15, 2007

The issue with investing

Let's say I invest in stock XYZ, and that because of my perception that in the future XYZ will be more valuable, I invest a sizable amount of money.

Then, let's say I become aware that XYZ is engaged in things I disagree with, but that regardless of how bad they are, the realistic observation is that they will still make money.

Do I pull the plug on my investment? Do I question XYZ's behavior in public?

And it's more than that. What about my car and my house, which are investments as well? And note I do not quite have much choice there: I must be able to afford those things. So... do I pull my part of the plug on the things that support my ability to have a roof? Do I question the system that is merciful enough to allow me a roof?

Hold on, it should be my right to have a roof! What is this charity I am supporting with my investing now?

That is the problem with having things. By virtue of the fact that you want stuff, it is way too easy to change who you are to support the status quo. The possibility of becoming a slave of your possessions, and selling your soul to stuff, is too great. And these days, we can easily see what happens when reality catches up with us.

So what do we do when we have a conflict of interest with the way in which we live?

Thanks for the shiny beads, but no thanks.

Tuesday, August 14, 2007

New papers available for download

Update: a new version of Instance Specific Behavior in Visual Smalltalk and Simulating Method Wrappers in Visual Smalltalk are now linked below. Make sure you have the latest versions!

So I got three new papers describing serious efforts done in Smalltalk. While the work was done in Visual Smalltalk, they illustrate very well how to attack development process issues using Smalltalk itself.

First, we have Simulating Method Wrappers in Visual Smalltalk and Instance Specific Behavior in Visual Smalltalk. Note that related topics were discussed at length over the IRC channel not long ago. These papers include several techniques to speed up performance. With these facilities in your system, you have a code coverage tool essentially for free, and you can also do things such as checking for stack overflow via SUnit without actually causing it (!).

Finally, we have An Efficient Difference Finder, which talks about how to use a code coverage tool made possible by the previous paper to streamline code merging. It is fascinating to see how the patterns I've seen work so well at other places are present here too. I guess there are only so many ways in which you can be productive.

The papers can be downloaded from ftp://sqrmax.homeip.net. Note there is no anonymous user, so you have to use the user name "smalltalkVideo" with the password "now". Also, be sure to use active FTP. Or, if you have trouble with the FTP server, use the HTTP links above.

Enjoy!

Update: the files are available at the FTP server now, user "smalltalk" password "now".

Hash Analysis Tool v2.0 is released!

Finally! The Hash Analysis Tool v2.0 is now released. The changes since the last beta are as follows.

  • Published the needed prerequisite, Integer>>hashMultiply, to the public Store repository. But note that in later versions of VisualWorks, this may become an unnecessary prerequisite.
  • Fixed a small problem with chi^2 calculations.
  • Added all the original experiments that were the basis for the Hash Analysis Tool v1.x as hash functions (see the string sparse sample hash functions).
Enjoy!

Sunday, August 12, 2007

Hash Analysis Tool v2.0 beta

The Hash Analysis Tool v2.0 has reached beta status. As a noteworthy addition, it now has word list data sets for 6 different languages built in.

Enjoy!

Update: I just published beta1 with an added button so you can inspect only large hash buckets. I do not think there's any feature left that it is worth adding at this time. If everything goes fine, I will move from beta to release shortly.

Strange relationship

So I was checking some things in the Hash Analysis Tool, and I noticed the following.

If the hash function appears to be of reasonable quality, then the load factor for size p is roughly equal to the normalized chi squared mod p test result plus the fraction (amount of objects minus amount of hash values) over (amount of objects).

What in the... I tried to get a proof, but got nowhere. Any ideas?