Tuesday, December 20, 2005

Regarding the New York transportation strike

I have heard several people make comments regarding the MTA strike. Most of what I listened to was essentially visceral rage. How dare they, fire them, etc.

I wish we treated people like the MTA managers that stole 1 billion dollars in the same way. I wish we still remembered who is Kenneth Lay with similar fury and indignation.

But we do not. And in that way, we seem willing to punish what we perceive as union greed more harshly than corporate greed which costs us much more than a walking inconvenience, or even money. Because the people that repeatedly lie about polluting our environment, the people that rip our pensions and retirements apart, the corrupt people we vote into office, and the people that send us to a war for no reason... none of those people receive the treatment we are dishing out today.

So my friend... I hope we remember this day and muster enough indignation to do something when the issue at hand is much more important. Because when we do not do that, we do not amount to much more than the hypocritical behavior we fail to condemn.

Saturday, December 17, 2005

What you get is what you create

I think that as pressure to ease our consumist tendencies increase, the overall quality of whatever pacifier we choose decreases.

Evidently, the pacifier companies want to sell more than they want to be helpful. This is because their own pacifier is money. Since we cannot do with a single pacifier for more than a relatively short period of time, then the quality of pacifiers past that critical time window is pointless. Consumers, addicted to pacifier-like substances, will buy another pacifier anyway.

Therefore, in general it seems that we have the expectation that we will stop using things rather sooner than later. Our possessions become our toilet paper. When manufacturers figure that out, they cut costs further by not designing their products to last. And therefore if someone comes along and actually demands quality, fine: sell them a multi-year extended warranty (which now looks like a repair warrant). Other than that, consumers will just go and get another one because they just can't help themselves, so quality does not matter.

I remember disgusting instances of this behavior that goes largely unpunished. Among my favorite examples, there is the Ford Focus 1999, which in its first year was recalled 11 times. One of those recalls addressed the issue of the car bursting into flames for no reason whatsoever.

I also believe that we consume much more than just things. In our lack of education, unable to think for ourselves, we also consume our own inner peace in terms of statements that we desperately want to believe in spite of any and all evidence against them, such as:

  • This rather ancient book says the world was created out of thin air less than 10 thousand years ago. Since I like the book even when it does not provide any proof for its assertions, thousands of years of scientific investigation must be made illegal because I do not like their conclusions. I know Truth, I am Truth, and therefore anybody that I disagree with must be wrong.
  • Clearly I am not going to hell, and therefore I have the moral right, privilege and obligation to persecute gay people because clearly they are going to hell. I have the power to force people to conform to my own image.
  • Poor people are poor because they want to, and as such they do not deserve assistance. I have the power to decide who is valuable and who is not. I help pruning the worthless people off the face of Earth.
  • I feel threatened, therefore I shoot. I have the power to decide who lives and who does not.
If someone came over talking like the italicized text, how long would it take for you to evaluate the individual as a psychopath?

I think that, in part, this kind of stuff happens because of our so-called education and cultural context. One way or another, we end up so deeply addicted to things that come from our environment that we completely dismiss the idea of creating something on our own. We want to live in the Matrix. We do not evaluate things like self in our mind, and we become a long procedural mess, full of inconsistencies, bugs and atrocities we tenaciously refuse to debug*.

This state of mind prevents us from realizing how valuable our own lives can be in terms of what we can create for ourselves. Thus, we do not give a rat's ass about anybody or anything either.

We suck.

*Note how closely this mimicks the situation of most software projects.

Saturday, December 10, 2005

sounds.google.com

Searching for stuff on the Internet is easy, as long as things have been labeled in some language. We can do text because it itself is written in a language. Images are more or less ok, as long as someone describes them.

But how do you search for the name of a song when all you know is how it sounds? Therefore, I think that when computing power doesn't have anything better to do, we should give ourselves the ability to search for sound by submitting our whistles and hums.

Thursday, December 08, 2005

So, watching the news on TV, eh?

The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. --- Isaac Asimov

Saturday, December 03, 2005

When all is privately owned

There is this argument that says that you can have all the free speech you want, but that you will be protected only against what the government could do. The typical example is that badmouthing your employer in public will get you fired. Let's extend this line of thinking and see what happens.

So... what will happen when all universities are private? You say something that is not in the interest of profit, and then you cannot teach? Or inconvenient stuff gets censored from the library? Or you cannot study at all? Or you are even punished with community service, expelled, arrested or prosecuted?

So... what will happen when all types of health care are private? You say something that is not in the interest of profit and you sign your death sentence? No transplants for you? No hospitals, no doctors, no anything?

So... what will happen when all types of law enforcement and military are either private or directed by private interests?

Really?

In other words, if you don't have money, shut up and be thankful that you have a chance of being a slave. And you better comply because if not the people with money have the tasers, the microwave blasting humvees, and who knows what else waiting for you.

The authority in this argument is given by money. How can we claim we have any sort of freedom when this kind of thinking goes unchecked?

When we go ahead and simply equate "poor people" with "bottom of cost benefit analysis curve", and "rich people having too much of a power trip" with "the masters of the Universe", I truly believe we really suck.

"A science is said to be useful if its development tends to accentuate the existing inequalities in the distribution of wealth, or more directly promotes the destruction of human life." --- Godfrey Harold Hardy

Wednesday, November 09, 2005

What a life of contrasts

I don't watch TV. I listen the best music I can get, by the best performers possible. But every so often, some bits of the crazyness outside my home reach my life.

For example, I received a lovely pamphlet from PSE&G, my electricity company, regarding the level of pollution in New Jersey. The graph doesn't make any sense! What in hell do 95%, 98%, 100% and 271% mean? Bullshit!

Here are some other examples of things that are 200% of nothing: clean coal, free trade creates jobs, clear skies, the last episode of so-called intelligent design, and this pervasive idea of persecuting anybody that does not behave like "us".

This crap reminds me of one time when I was at Omaha's zoo. There were a number of gorillas in a lexan cage (it was winter and they kept them indoors). One of them was lying down on the ground looking at people who were staring at her. There was also a woman by my side. All of a sudden, she asked me:

Do you think they realize we are here?

At first I thought the lady's brain must not have had any ECC, because clearly the lexan was transparent. My first reaction was to find a section of the rounded cage where you could see through two layers of lexan, thus proving beyond any reasonable doubt that the lexan was not mirrored on one side. As I found such a place, it hit me. That was not the question the lady had asked.

Humans suck man, they can suck so bad it's sickening.

However... if you want to see, if you allow yourself to use your eyes, amazingly, you can see.

Inside the huge cage, there is a subcage where the gorillas are fed individually. One of them, apparently a she, was eating. But... strange... she was eating very slowly, and looking to the side most of the time. What was going on? She was showing how she was eating to another gorilla outside the feeding cage, who was evidently more than furious and green with envy at the situation. This one appeared to be a him. His face was so similar to what you could see from anybody in that situation... amazing... I wish I had had my video camera at the time.

A few months later, I was looking at the caged gorillas again. This time, they had a silverback male who was having the time of his life chasing all the other gorillas down every so often. According to the plaques, this is his typical behavior through which he displays his dominance. He would also bang his huge knotted fist against the lexan, making quite some noise.

At one point, he started chasing down another gorilla running parallel to the lexan wall. I don't care the lexan is there --- the sight of this enormous gorilla running towards you makes you get out of the way.

From a more removed point of view I looked at the scene for a while. The silverback would mockingly charge every other gorilla more or less in turns. The others would move away a bit in a display of subordination. And the thing would go on and on...

I was about to leave. I was at the edge of the cage, where the lexan and the cage wall meet, and one of the persecuted gorillas came my way and sat down on a fake rock after displaying subordination to the silverback. But he looked at me. And I looked at him. He was curious. I waved at him. But he was still curious. I had my camera bag with me, and I had just put my camera away. Oh...

I opened the camera bag and started showing him my camera, the telephoto lens, the rolls... and once every few seconds, he would look back to see if it was his turn to run away from the silverback. Eventually the moment came and he ran away.

But now I was left with my open camera bag, some camera stuff in my hands, and a big ass silverback staring at me with disapproval. He was more than pissed. How had I dared entertain one of his subordinates!!! He had such a human-like expression of "if it weren't for this lexan, you would be very sorry now".

I started packing my camera feeling I should hurry up. But before I could leave, the silverback started chasing another gorilla... and as he left to perform his duties, he banged the lexan next to me with his huge knotted fist.

Exactly why is it that we think we're so superior, now? I truly think some people should be in zoos, and that we should let these creatures live in peace in their own piece of wilderness.

Sunday, October 30, 2005

Hydrogen, schmydrogen

It's fine, after all. Let's continue this gas price crazyness, this record profit setting thanks to our inability to kick the dope of gas guzzling, because we will have hydrogen in the future.

Yeah, sure. Never mind where are we going to get the energy to obtain the hydrogen in the first place. Never mind where are we going to store the necessary amounts to guzzle like we do now. And by the way, I refuse to drive around FAE bombs waiting for a crash to explode.

For a long time I wondered how was it possible to continue to push hydrogen without looking at any other alternatives when clearly there are so many hurdles on the road. But then... maybe I'm wrong, but now it seems to me it's a stalling tactic.

In other words, let hydrogen fail. It will take a long time to do so, but until it does we will have an excellent excuse to avoid investing in anything that could possibly work. Meanwhile, oil prices will skyrocket to 5 figures per barrel, which is what some haves and have mores favor anyway.

Man... are we stupid or what???

Saturday, October 29, 2005

Distinguish for yourself

Want to test how many distinctions can you handle at the same time? Do you want to check how fast you can distinguish for yourself? Then check this game out.

TARFU

Dude... things are really messed up. Did you know your printer could have been marking all the printed pages with information that could identify you? Check it out, utterly disgusting.

I am not buying a printer any time soon. Are ethernet cards spying on us as well? Screw this bullshit man!

Friday, October 28, 2005

Proper abuse of language

The last year plus has brought a number of changes. I am now in an environment where I can more easily share the fun I have when I do my job properly. And since 99% of the time I am in a good mood, phrases that reflect the fact come to me... and what's even better, others start using them as well...

Almost certainly, my most infamous hit so far is "my little friend", said with an obviously fake accent. At the office, my friend, people are starting to copy even the accent! It's a lot of fun and it immediately causes smiling by association. Working in a good mood with smiling people is very pleasant!

What's even better, other people start creating new language as well. Among friends, I have no problem describing bullshit as bullshit... but again, with the fake accent so it's clear it's joking in a good mood. Based on the fake accent, one of my colleagues gave me this lovely present:

boolshit = bad logic

Hilarious! Absolutely fantastic! It should be used more often!

The doorman just called me on the phone to tell me "ah my friend... you have a delivery my friend... have a good night my friend"... and yes, it is dinner time so catch you later.

Monday, October 17, 2005

What an amazing picture!!!

And it is for real --- check it out!

Monday, October 10, 2005

Extremely disgusting achievement

Oh, Apple, what a lovely patent. To think that I considered letting my next computer be a Mac... just pisses me off.

Sooner or later a corporation will be granted a patent for creating human beings followed by people themselves being considered copyright violations.

I shall continue listening to Borodine's In The Steppes of Central Asia at this time.

Sunday, October 02, 2005

Consumer requirements

I was given an old Apple Powerbook 190. It has a black and white LCD display, a ~400mb hard drive with about 150mb free, 16 mb of ram, a floppy drive, and a touch pad. And it works. This could have been quite a nice computer some years ago.

But let me tell you what I found while I played with it. First of all, the operating system uses 4mb of ram to run. Doesn't that sound funny already? And it is also quite fast to open programs and such, especially because just a select few are larger than 500kb. With a setup like that, you don't even need virtual memory.

What's wrong with this computer? It can't run the latest applications, essentially. Because if I wanted to write, I could easily do that. I'm not sure about Smalltalk, but there must be one that runs in this machine.

So who changed our requirements such that we're not happy without much more? We were satisfied with that a few years ago, right? It's only by comparison that we think we can't do without the new.

I think that, sometimes, the newest stuff is designed to keep us occupied using the new features instead of significantly improving or changing what we do. There is too much of "buy, and then throw in the garbage 5 minutes later" going on.

But what about using your life to actually create something? Because evidently you can readily get yourself occupied with new features like everybody else. It is not unique nor special whatsoever. What about producing a valuable personal expression of the world as you see it?

Quite unfortunate, our situation as easily-duped consumers.

Saturday, October 01, 2005

Good dealio!

I received a check from Don Knuth today. He also sent me a hardcopy of my modest suggestion for The Art of Computer Programming with some notes in his own handwriting. What a nice guy!

I feel quite strange, but good at the same time... very very weird. In any case, now I need to do more of my number crunching!

PS: the check number is 256, and it's for 32 cents.

Friday, September 23, 2005

Very nice methods

Sometimes you just want to do something for all subsets of a particular collection (including the "empty" and "all" subsets). The typical stuff that comes to mind is to count from 0 to 2^collection size - 1 (heh, or collection size mersenne), then sift through the bits and add stuff at the proper index when the proper bit is 1, etc. You get the idea.

The other day I needed to take the products of all possible subsets of a collection of integers. I had already implemented the bit-sifting approach when I realized I had sinned against my own principles by having been too clever when it had not been necessary.

So I dumped all that bit-sifting code and, instead, followed what we actually do. Consider the collection $A to: $Z, and all its possible substrings (e.g.: 'ABCZ', 'JRTXY', 'F', etc). Clearly, all the substrings of $A to: $Z are all the possible substrings of $B to: $Z, once prefixed with $A and once not prefixed with $A.

The same argument applies all the way down to $Z to: $Z. So all the number iteration becomes process stack recursion in terms of sending messages, without any of the bit-sifting arithmetic nonsense.

So, for my product problem, I implemented these methods in SequenceableCollection:

    allProductsTimes: aNumber
    do: aBlock

      self
        allProductsTimes: aNumber
        startingAt: 1
        do: aBlock

    allProductsTimes: aNumber
    startingAt: anIndex
    do: aBlock

      anIndex > self size
        ifTrue: [aBlock value: aNumber]
        ifFalse:
          [self
            allProductsTimes: aNumber
            startingAt: anIndex + 1
            do: aBlock.
          self
            allProductsTimes: aNumber * (self at: anIndex)
            startingAt: anIndex + 1
            do: aBlock]


Isn't that absolutely beautiful?

Since the elements of the collection could be polynomials or matrices, then I can't assume the empty product is 1. The first method used to refer to the value of the empty product. But since the second method multiplies everything that follows by something that is given, these new methods ended having more functionality for free. Fantastic, fantastic.

Thursday, September 22, 2005

4 to the millionth power --- behold!

Please take a look at this. Come on, man... 4 to the millionth power isn't a challenge for a base-2 computer:

4^1,000,000 = (2^2)^1,000,000 = 2^2,000,000

or, equivalently,

1 bitShift: 2,000,000

Suggestion: try 1023^200,000 instead.

Sunday, September 18, 2005

Newton-Raphson common inefficiencies

If you need to calculate sqrt floor, then typically the Newton-Raphson algorithm will give you a proper answer very quickly, with the following words of advice.

Typically, the first value for the recursion is set to something like self // 2. But, my friend... that is not very insightful. If self = 2^1000, then the initial guess is set to 2^999 instead of something much more reasonable like 2^501. In that case, poor Newton-Raphson will have to burn through unnecessarily large integer multiplications and additions just to get to 2^501, from which point the real work begins... sigh... so please, use 1 bitShift: self highBit // 2 + 1.

I didn't suggest 2^500 as the initial guess because Newton-Raphson works by approximating from above, not from below.

Interestingly, for small large integers, in VisualWorks it's faster to find bits of the square root by using the fact that, for h = 2^n,

(r+h)^2 = r^2 + 2rh + h^2

can be calculated without any multiplications whatsoever (you can initialize r = 1 bitShift: self highBit // 2, r^2 is obviously 1 bitShift: self highBit - (self highBit \\ 2), and then you can remember the new r^2 = (r+h)^2 from the previous step). Eventually, however, Newton-Raphson becomes more efficient.

But if you are in a real hurry, then by all means use that floating point processor you have. Take the 50-whatever most significant bits of the large integer, convert them to a double precision floating point number, obtain a square root for that, then shift the answer back into place and use that value + 1 for the initial guess in your lovely Newton-Raphson implementation. Then Newton-Raphson won't have to burn through large integer arithmetic to obtain 25-plus bits of the correct answer --- plus in the next iteration, it will duplicate the amount of correct bits...

This means that since SmallIntegers fit in a double precision floating point number, Newton-Raphson should be implemented for LargeIntegers only.

Friday, September 16, 2005

Creative (at least for me) use of exceptions

So... number crunching... one of the things I got done before I got this ugly flu was a sieved Fermat factoring algorithm. Essentially, if N is odd,

N = pq = (x+y) (x-y) = x^2 - y^2

And if you set m = N sqrt floor, then

N = m^2 + r

If you require than gcd (m, r) = 1 to detect trivial factors, then 0 < r < 2m because of m's maximality. Since we need to find pairs of x^2, y^2, we can increase m by j and operate. Thus,

N = (m+j)^2 - (j^2 + 2jm -r)

Getting bounds for j is not hard based on the difference between (x+y) and (x-y) --- it can't be just about anything. So the thing is, within the range of possible values of j, to find values of j such that, for some k,

f(j) = j^2 + 2jm - r = k^2

Clearly, you look at f(j) mod some primes and determine which values of j cause f(j) to be a quadratic residue mod some well-chosen integers. The choice of moduli is too much to write about now, but there is a closed formula for the amount of quadratic residues mod x based on the factorization of x, and then you can see which moduli are most effective based on the ratio between the amount of quadratic residues and the modulo.

Anyway. Let's say you start with 16, and you find that j = 0, 3 and 6 mod 16 cause f(j) to be a quadratic residue mod 16. You eliminated 13/16 of the total candidates. But there are still so many values of j that you need to sieve further. So you choose, say, 27, and try again. But now you only try numbers mod 16*27 which are 0, 3 and 6 mod 16 to begin with. And so on, until that particular line has just a few possible values of j, at which point you brute force them by checking if they are squares for real.

Before, I had chosen to reuse my SchedulingSetQueue, so the thing would generate tons of modIntegers which represented some possible values of j. But the thing was that the size of the queue was out of control, and the possibility of duplicates forced me to keep all the modIntegers seen by the queue in a Set. A ton of work!

So, since I was caching all the modIntegerRings with their quadratic residues anyway, I thought I could replace all that recursion flattening with a stack recursion strategy and make it go faster.

Faster indeed. But I had to overcome a problem. Once the sieve finds a factor, it needs to kill the whole recursion and exit. And of course, the recursion was implemented in terms of several methods. There was no single checkpoint where I could write the fast-out code. And passing around the necessary information so that the recursion could unwind itself felt disgusting.

Solution? Start the recursion with on:do:, and watch for SievedFermatShouldStopEvent notifications. If such a thing happens, ex return: nil and that's it.

Fantastic! So I can just write code as if the normal case is all that happens, and break right out of it by means of signaling an exceptional event such as "factor found --- stop!". So clean, so nice!

Thursday, September 15, 2005

asm in Smalltalk

I'm back into number crunching. One of the things that I thought would be useful for some situations would be the ability to write pseudo-asm or C inside a method, and let some GCC-like compiler behind the scenes do the dirty work of producing native code out of that.

I don't mean Slang. I mean something like {c code statements;} or {assembler statements} right inside the method, with the compiler figuring out all the name references.

I don't like the separation between C-world and Smalltalk-world. I don't want to switch to another editor, write some stuff, then try to make a VM plugin that doesn't blow the whole thing away because of some call protocol that has a bug, etc. I do not want to see the compile-try-crash-debug cycle again. Why can't just I write the C code I need in a method, test it with SUnit, and if it works then all is good that's it?

Sunday, September 04, 2005

NOLA --- open your eyes once and for all!

Does it take writing stuff like this to make others kick the dope of arrogance? Seriously guys, it's very very late and it's time to wake up already, ok?

I don't think people have realized what this disaster is yet. It reminds me of the large groups of dumb mammals which watch their kin being eaten by the lions from a safe distance thinking it's fine because it wasn't them.

How disgusting.

Large folders in NTFS, part 2

A while ago I commented that, on my fast hard drives, reading an unfragmented folder with about 55k files took over a minute the first time, and 17 seconds the second time after everything was cached.

I subsequently burned all those files to a DVD. Reading that folder from the DVD for the first time without any caching whatsoever took less than 10 seconds.

There must be some scenario in which the current NTFS strategy wins, but... hard drive vs. DVD and the optical drive wins, even giving the advantage of everything cached vs. first read? Is NTFS so featureful that is has to be slow?

So my friend... NTFS for file server work? You have to be kidding me.

Wednesday, August 31, 2005

Under stress, the inner self flourishes

I just read this regarding New Orleans:

"It's like being in a Third World country," Mitch Handrich, a manager at Louisiana's biggest public hospital told the AP. <link>

So all of a sudden this kind of situation is like being in a third world country. Because, you know, just imagine: any given third world country is like New Orleans now. It is so bad, so unbearable, it doesn't even have air conditioning! How ugly it must be to live in a third world country!

Man, what an unfortunate, uneducated, insulting, and bigoted comment to reproduce.

And this bugs me even more because this was uttered in a country that, unlike poorer regions of the world, has always had the resources and the opportunity to address issues like those that affect New Orleans today --- and in this case, with more than plenty of warning before the disaster happened.

Just like with what happened with the Shuttle. Fifty-plus flights with foam problems? It's so easy to think of Dilbert's manager going "la-la-la-la"... until Columbia is lost.

Then what do you say? I screwed up big time, therefore I quit? Of course not. Just pull emotional chains to save yourself. Poor astronauts!

Same deal here. Even when New Orleans has been flooded more than once already, even when everybody knows what will eventually happen, the decision was made to install a half-good solution.

Isn't a city with more than one million people worth a few more feet of reinforced dirt in the right place? Isn't it worth it even if you want to look at it cynically, from the point of view of the oil facilities?

What I hate the most is that amidst all these horrible yet avoidable losses of all kinds, there is zero accountability. Wake up and realize that some people put everybody and everything on the line, against available evidence placed a bet saying there would never be a higher-than-category-3 hurricane in the area, and lost it all!!!

When you deny the facts and make crap up under the influence of arrogance, you get stuff like Columbia and Chernobyl, the Kursk and Challenger, global warming and nuclear winter. We are much more humble and modest than we think. Now face reality and deal with the consequences.

Let's make sure we remember our lesson and build the thing right next time.

Monday, August 29, 2005

Night full of art

See this is the kind of stuff I really like... to enjoy the art of the masters, not the students.

First, I listened to Prokofiev's Peter and the Wolf, this time with Spanish narration. Very very nice. What a masterful arrangement of instruments, rythm and melody. Fantastic, fantastic.

Then I watched, for like the third time, a movie that I cherish very much: Being There. What an incredible performance by Peter Sellers. What a fabulous portrait. I wish there were more movies like Being There, or The Party.

As I look back at the movies I really like, I see that it is usually the case that I like the performance of artists who usually perform in a theater. How would it be possible to compare any of the so-called good actors of today with someone like Peter Sellers? There is no point in trying, in the same way that Karl Leister makes any "good" clarinet player sound bad.

Now I remember something from a short story called "You will have to forgive me". It says that time is unfair in that it does not stop when it should, and that it has a tendency to break the perfect moments and to avoid preserving things and people at their best.

And yes: how unfair that the production of the great people that have lived on Earth is so explicitly bounded by their lifetimes, and especially so when we consider how little we usually do to support creativity. In that regard, we should consider these artistic expressions as valuable as a fortune because they are so much more than the random noise we are fed everyday.

Thank you so much Peter.

The master of the clarinet

And I thought I had heard good musicians playing clarinet... no way! If you like the clarinet, then there's one dude you have to listen to: Karl Leister.

How is it possible that he extracts such stable, perfectly defined and colorful notes out of his clarinet without any spurious sounds? All the time? I didn't pick up a single glitch! Amazing.

Good stuff, Karl, really good stuff. Thank you so much for all the effort you must have put towards playing like that!

Sunday, August 28, 2005

Well, excuse me!

So exactly why should I shell out tons of money in a new powerful computer I need so badly when, while cpu shopping, I discover this article? In short:

The Pentium M runs cooler, draws much less power, is faster, and is cheaper than most state of the art x86 cpus!

And with the help of an adapter, you can stick this lovely cpu on some desktop motherboards, even overclock it a little bit and... wow. Take a look at the article.

I knew P4s weren't that great. I was hoping AMD processors would be significantly better. But no, none of the above, because a slower P3 on steroids running on slower memory is better than both!!!

That super-size-everything dope really bends our perceptions out of shape, man...

Sunday, August 21, 2005

Games people play

I play computer games. Two interesting things happen.

I only play games which I've had the patience to master. This usually takes me considerable time, so I am picky in what I invest my time. This means I play few games.

The other thing is that, after I've had the patience to try to master the games, I only play games in which I enjoy watching myself play as I play. I become my own spectator and my own fan. And... as with great moves you always remember from your favorite sports stars, I also remember my great moves.

For example, how could I forget that amazing game against that dude "Petre", who had the annoying habit of insisting I played a soccer game rather badly? I finally convinced him to play against me. I even played on the right-to-left side, which is more difficult since you usually play left-to-right. I was winning 1-0, and at one point I found myself in what seemed a dead position. He even asked me "now what?". And then I invented something new. I created a play out of thin air for the 2-0 lead. His friends, behind, grew more silent. But there was more.

I got the ball close to midfield with #8 and ran away, his #8 player running behind mine. And I did many things I never do: I started describing the play aloud, how #8 was escaping the other #8, how mine stopped to raise the ball in the air by tapping it from below a couple times, and how he kicked the ball towards the goal from far away, all calculated so that the other #8 could not do anything but watch.

Players know from experience that trying to score this way against a good goalkeeper is useless. That's why it's not even attempted. But the ball went and went, through defenders and attackers who had been taken into consideration, and all of which had to watch it go by. The goal was not in the screen for me to measure the shot, but I didn't need that. Because the ball went right through this very narrow gap between the stretching keeper and the post from what would have been 35 yards out.

And then the silence, the faces of amazement, the gestures of astonishment, mouths half open and eyes staring into space. These were guys who knew the game, and they could not believe what had happened. My own spectator couldn't either. Fantastic.

I play for moments like this one. Not because of the defeat of others or the computer, but because I enjoy watching myself create. I remember many others moments very well. Petre's 3-0 subsequent loss on the second match. A 9-3 win on aggregate (6-2, 3-1) against another dude who knew the game very well. The victories with bad teams, against all odds. And that -19 round of golf.

Smalltalk is more of the same for me. I spent a lot of time perfecting, distilling the style. After the initial disappointments and the times when I wouldn't enjoy myself, came the time of my own spectator's approval and cheering because of what he was watching.

This led me to think this could be why people who program in more mainstream languages find Smalltalk so strange and foreign. Because of its tendency to give you very clear feedback on what you are doing, mastering it requires an effort for which our culture does not prepare us. And since we're not prepared to understand it, it is difficult to enjoy it.

There are so few development environments, so few instruments as expressive as Smalltalk, out of which you can extract so much by doing so little with so much elegance. How unfortunate we are conditioned to become First Triangle instead of First Violin, or an expressive Piano Virtuoso. What a loss to all of us.

Saturday, August 13, 2005

Large directories and NTFS

Use case:

  1. Create new directory.
  2. Put on the order of 55 thousand files in it.
  3. Do dir directoryName >someTempFile.txt
What, you expected it to finish quickly? I don't think so. On this machine, with a fast hard drive, it takes several minutes.

Excuse: it's because the directory is fragmented. Good observation! Chances are that directory is big time fragmented now. Solution: defragment the whole drive and try again.

It still takes over a minute.

Excuse: it's because the hard drive does not have anything cached after that defragmentation run. Ok, fine. Solution: try again immediately so everything is still cached.

It still takes 17 seconds.

No more excuses now. For extra pain, try to add a new file to that folder. The time it takes is much longer than a blink.

Think of the implications for file server work. It means that a hash-bucketed folder approach will be way cheaper than a large folder, even paying the price of having to hash all filenames to find them.

And the slowdown doesn't occur all of a sudden, no sir. It will creep in, hiding from plain view until it's too late. Put 1000 files in a folder and you will notice that adding files isn't fast anymore. Remember all those dollars you spent on the newest hardware upgrades? Well, by 5000 files, you will have remembered 300bps modems.

Quick: which are the largest folders in your system? Really? Does it hurt now?

So in the extremely rare cases when folders larger than "small" change, users should implement a hash-bucket folder structure because the OS is unable to deal with the exceptional case efficiently.

BTrees sure sound cool now...

Monday, August 08, 2005

Mr. Franz

Franz Schubert lived for a very short period of time, dying still quite young at age 31. But he still managed to compose 1,000 works! How in heck did he do it? How fantastic. I wish today's world was more supportive of people like him.

I remember The Shepherd on the Rock (D. 965) very well. My mom (piano) and my uncle (clarinet) played it a lot at home, rehearsing for concerts with my mom's repertoire students...

Saturday, August 06, 2005

Desensitizing the public opinion

If we didn't have all the cultural baggage, both spontaneous and manufactured, how would our typical TV broadcasts appear to us?

For example, here's the 60th anniversary of Hiroshima's destruction. Do we ever stop to contemplate the fact that a rather "small" bomb as atomic bombs go managed to vaporize on the order of 100,000 people, mostly civilians without military value by the way, in about the time it takes to blink? And after doing it once, then we go and do the same thing with Nagasaki.

But here is all this programming talking about the honor of the air force pilots, or the sense of duty to go vaporize civilians, about how a couple survivors dealt with it... as if it hadn't been that bad after all.

Take a look at Hiroshima after the bombing, and pay special attention to the scale. This is the real impact of the first atomic bomb some people seem to be so proud of. I do not know how is it that we even dare to speak about "ground zero".

I really hate the fact that we miss the point here. It's not about "oh, we saved our soldiers' lives", because Japan was more than defeated already. We committed two horrific acts against civilians, comparable to what happened in concentration camps for some weeks but distilled down to a fraction of a second. And since we took these extremely cheap shots on them and it didn't happen to us, it's ok somehow.

If we had really wanted to win, it would have been much better to take out the leadership. But after all those hundreds of thousands of civilians vaporized, most of their leaders were still alive. Those that were powerful, those that encouraged crazy acts, were still alive so they could be embarrassed and humiliated by us: the arrogant people with big bombs and a Dukes of Hazzard mentality.

How about those lovely documentaries about the Bismarck and the Hood? Never mind that people on both ships wanted to gut each other out without anesthesia when they were shooting each other, as pathological such an artificially implanted knee-jerk reaction may seem. Never mind that those ships were purchased with tax money. Because you have to feel awe, you have to worship those killing machines we so proudly built. And while you sit on your leather couch, you have to pay respect and learn to justify how necessary is that those big gun muzzles shoot their deadly juices. You must be inspired by the honor of those chosen to use the long projectiles to penetrate others and inflict pain. Right?

The mentality of those programs is so sick, so beyond screwed up, that I can't completely describe how much it is disgusting to me.

Back at home, how about those "manuals to deal with radioactive fallout" given to civilians in Las Vegas back in the days of nuclear innocence? I also heard, from second hand sources, that if you take a laptop with you and drive towards certain places in the state of Nevada, the closer you get the more often the computer will crash. Is that nuts enough for you?

We're not at a time when stick'n'stone management will do. We proudly proclaim we have the largest geni I mean missiles and bombs. Like the chimps we are, we can mou I mean bomb anybody we want! We are the alpha male!... the point we miss is that all those weapons are regularly used to kill people that for the most part couldn't care less about what we do with our pretenses of democracy.

But our so called leaders, both politicians and shareholders, do all they can to drive us into believing this is normal, that there's nothing to worry, that it is us who are right when everybody else is wrong, and that everything we do is perfectly fine because it is regularly rubberstamped in the heavens of choice.

And regardless of the physical location, we answer the call to behave like this. Like the docile yet deceiptful sheep we are, and subjecting our own humanity to death by self-inflicted torture, we listen to what they have to say with the intense desire to be duped, to have an excuse to believe in something that deep down we know to be false, to be accomplices of the crimes we are about to commit against some vaguely defined "them" for our own benefit, and to let others be blamed when the truth comes out and a scapegoat is called for.

And our kids and our so called moral guardians, unable to do math and think properly for the most part, will decide our future.

As much as I cherish the product of real artists, as much as I think being aware of our place in the universe is extraordinarily precious, I just don't think we can deal with the responsibility that comes with the horrors we regularly inflict each other. We deserve evolution's axe. We're so much asking for it, it's almost pathetic we haven't wiped ourselves from the surface of Earth yet.

The final call to our self-destruction is being announced on the speakers. And as if we were invited to some fantastic party, we behave like lemmings and other animals we usually regard as stupid. We are unable to help eagerly jumping on board with excitement, feeling full of anticipation.

This, my friends, is bullshit.

Thursday, August 04, 2005

Just in case it wasn't clear already...

Take a look at this quote (and follow this link for the whole article):

"I found that if you made men more insecure about their masculinity, they displayed more homophobic attitudes, tended to support the Iraq War more and would be more willing to purchase an SUV over another type of vehicle" --- Robb Willer

Would we dare to look in the mirror and perform this kind of study on developers while measuring source code traits? We could induce intellectual prowess insecurity and see what happens.

In that scenario, how much could unnecessary complexity be seen as a compensation mechanism to deal with accomplishments that appear smaller than they ought to be?

Sigh... such a sad state of affairs.

Wednesday, August 03, 2005

Headphones

I listen to my favorite music many many times. I take it apart, I study it, I play it slower and faster than it really is, and I memorize it down to the most subtle details. I wouldn't be able to do it without a decent pair of headphones.

When I was younger, I used a pair of good Aiwa headphones until the head band mount broke down due to wear. That is very unfortunate, because those headphones had an aluminum transducer that sounded really really good.

After some time, I started hunting for a decent replacement. After finding that everyday electronic stores did not carry neither knew anything about what I was looking for, I landed in a pro music hardware place with a cd full of my favorite music. I asked, quite directly, for the real set of headphones. The guy immediately reached for this almost funky looking pair and told me to stop looking around, that I had found what I was looking for. The thing is that they were about to close, but the guy was nice enough to start up his personal cd player and old amplifier from the back room so I could give them a try.

He warned that it wouldn't be the ideal setup because the hardware wasn't that great, but that it would give me an idea of what they sounded like. So the cd starts playing and to my surprise, I couldn't recognize the music!

All this time I had listened to this stuff and hadn't noticed that? Or the arrangement in the background over there? All the instruments were so clearly separated by this fantastic piece of hardware that for me, those five minutes I was afforded to test them were a one way trip. I left the store without them as they locked the door behind me.

But that weekend when I got them... that meant a rediscovery of all the music I had listened to for such a long time with lousy equipment. All the richness hidden in what I guess was double-figure total harmonic distortion came shining out of the mud, pristine and crystal clear.

Not only that... but these headphones are also the most comfortable I've ever worn. I can use them for hours and hours, and even with the arms of my glasses behind my ears, they don't hurt or squish one bit. Their adjustment mechanism is brilliant, because they have no mechanism at all. You put them on, you make yourself comfortable, they adjust themselves to you, and that's it. Great stuff!

So if you are into headphones, please... look no more and get yourself a pair of AKG 240s.

Sunday, July 31, 2005

On hash

I think I should write this piece at least once.

There is a notion of equality between two entities in many languages. In C, for instance, one compares integers using the == operand.

The degree of freedom to compare two things is much greater in Smalltalk, since at least in principle you could compare whatever two objects you want. The method #= serves this purpose. Sometimes you may need to make sure that two names reference exactly the same object, and then #== serves that purpose.

As far as #= goes, consider then what happens when you choose one object. By means of #=, you have implicitly chosen all objects equivalent to that object. In this way, the object you chose is the representative of all the objects which are equal to the one you chose according to #=.

Using basic algebra classes, you can see that #= is an equivalence relationship, and that #= partitions the space of all objects into classes of equivalence, each of them being a collection of objects that are equal to each other according to #=. All classes of equivalence are disjoint, since as soon as they have a non-empty intersection, the viral behavior of #= joins them completely.

Consider now a Set. Sets use the method #= to partition objects into classes of equivalence. They will hold up to one representative object for each equivalence class. Hence their well known characteristic: they will hold only one occurrence of each object.

This is because you will have A = A. Suppose the set S already contains A. When you try to add A to S again, it will find it already has a representative of the equivalence class of A. And it will know this because it will find, through its contents, one object equal to A --- in this case, A itself.

Think of the implementation details though. If you have a large set, searching for representatives in a linear fashion will take a long time. But this would not take advantage of the fact that no two objects in the set are equal to each other.

Sets usually implement object storage in terms of an array. Let's say you had a way to give a more or less unique name to each equivalence class. You could easily map those names into integers, and then store each representative in the storage array at the position dictated by those integers.

Sets obtain such integers from objects by sending them #hash.

Note that since Dictionaries keep an implicit set of their keys, they also depend on #hash.

Let's examine the properties of this hash function that takes an equivalence class, gives it a name, and answers a more or less unique integer.

First of all, why "more or less" unique? Because even with simple objects, you can have a huge number of equivalence classes. This amount can be so large that it frequently makes the number of particles in the Universe, roughly 2^266, look insignificant.

Consider for example the set of all ASCII string of size 256. There are 2^2048 classes of equivalence because there are 2^2048 possible strings. Chances are you won't need (and certainly won't be able to store) all possible strings at the same time.

A compromise solution is usually taken: equivalence class names will be mapped to integers in the positive SmallInteger range. This provides room for something on the order of 2^30 possible distinct names. Another advantage is that this limit also keeps arithmetic in the SmallInteger range, where it is fastest.

Let's examine a straightforward way to implement #hash given an implementation of #=. Typically, #= will compare the values of some instance variables. These are the ones that determine the equivalence class the object belongs to. Since most of the time we can delegate #hash to those instance variables, all we need to do is to combine their hashes.

At first we may be tempted to sum all the hashes. But that may result in LargeInteger arithmetic. Since fixing that takes ifTrue:ifFalse: usually after the damage is done, addition is not a good idea. The logical operations #bitOr: and #bitAnd: tend to just turn all bits on or off, respectively, so it's not a good idea either.

The logical operation #bitXor:, however, does not have a tendency to either all ones or all zeros. Using it to mix hash values is fine, with an important assumption: the values hashed should already be in the SmallInteger range.

This is because #bitXor: outputs a result of the same size as the arguments. For example, if it was used to hash a byte array by combining the hash values of the bytes, the result would result in an integer between 0 and 255 --- too constraining. These situations will be discussed in more detail in a separate post.

So in situations where you can obtain SmallInteger range hash values from the instance variables, #hash can be implemented by mixing those hashes with #bitXor:. For example, if you had 3 instance variables, you could let #hash be implemented in terms of ^(iVar1 hash bitXor: iVar2 hash) bitXor: iVar3 hash.

The hash functions that map an equivalence class name to an integer must also comply with this requirement:

x = y => x hash = y hash

This is clear now, because if x = y that means they belong to the same equivalence class. This equivalence class can only have one name, and the hash function will obviously map it to a single, unique integer.

Note that nothing can be said when x ~= y. In that case, you may even have x hash = y hash. This happens because of the size restriction applied to the integers given by #hash.

When x ~= y but x hash = y hash, it is said that the hash values of x and y collide. As far as Sets are concerned, this may cause a slowdown because they will have to resort to storing representatives in array positions reserved for other representatives. It is still the case that this will be faster than linear search for the vast majority of cases, but it should be avoided.

A hash function is good when it produces few collisions. Few collisions are good because they allow sets to be fast.

However, we will be constraining hashes to the SmallInteger range. So when #= partitions objects into a LargeInteger amount of equivalence classes, we will have collisions for sure. How can we obtain a hash function that is good then?

The key observation is that we will usually deal with far less objects than the maximum SmallInteger value. In practice, however, this implies that it will be pretty common to hash things that are similar in structure. Names of up to 32 characters, for example, are an extremely tiny amount of all possible strings up to size 32. So in general, hash functions will have to be good even when hashing things that have a very constrained shape.

The arguments above lead to our first rule of thumb: #hash should produce different results when the objects hashed are similar.

This rule of thumb has a powerful implication. In the absence of any knowledge about the structure and characteristics of the objects that will be hashed in practice, there is no other choice than to hash everything that is referenced in the #= method.

More concretely, sometimes the hash of collections is implemented in terms of hashing a few selected objects, typically the first one, last one, and a few from around the middle. This may seem clever, however note that it violates our rule of thumb. And because of this, hash functions implemented in this manner will produce lots of collisions when the inputs tend to be similar at the indices used to calculate the hash.

This is more common than it may seem. For example, you could consider a LargeInteger as a collection of bytes. Thus, the hash of a LargeInteger could be the hash of a few selected bytes. This was the case in Squeak Smalltalk, and it was just fine for a long time. But when Magma, a relational database written in Squeak, used LargeIntegers as pointers, it so happened to be that pointers were usually equal at all the cherry-picked locations. Hence, the hash of most pointers was the same.

This problem could have been fixed by implementing #magmaHash and subclassing Set (or Dictionary, which keeps a set of keys using #hash). But then things get uncomfortable. Because if #magmaHash is better than #hash, then why shouldn't #hash be replaced? And if this is the case, then it would have been preferable to let #hash go through all the bytes to begin with.

This raises the issue of performance when hashing large collections. The solution to this is to implement #applicationHash using the knowledge about what is being hashed. This can be done because the knowledge is known in the context of the application in question. And in fact, such a hash could be even more efficient than a generic, cherry-picking hash --- it could be the case that less cherry-picking is enough.

Since #applicationHash would be written with specific knowledge of the application in question, this solution is significantly different and better in quality than implementing a cherry-picking #hash and forcing everybody else to write a better, all-encompassing #hash when excessive collisions become evident within their application. In every case, #hash would produce quality results which would keep the performance of collections that depend on #hash at a near optimum value.

In short, here is our second rule of thumb: hash the whole collection, unless it causes performance issues. Only when this is an issue as shown by a profiler, resort to specific knowledge to implement an #applicationSpecificHash in terms of fewer, significant elements of the collections in question. You should make sure that this new hash is good in terms of collisions.

With this in mind, note that some Smalltalks already come with pluggable-hash collections, in which the implementation of #hash used by the collection can be provided by a block. Depending on the Smalltalk you are using, you may even be able to skip subclassing Set or Dictionary so they use the application specific hash method.

This leaves addressing the implementation of #hash for collections that contain objects whose hashes are heavily constrained. Examples of this would be the hashes of byte arrays, strings, and integers. I will address these in a separate post.

Saturday, July 30, 2005

Brave new world

So did you read that book by Aldous Huxley? I just read the first chapter. I don't know if I will have the patience it will take to read it whole. Because what Huxley is describing is happening right now under our very noses: the conditioning, the dumbing down of our culture, of what we think is valuable in life... anything, whatever it takes as long as we like being the slaves we are.

And what about the really important issues? Just a deafening silence.

About stuff for kids

Maria Elena Walsh is one of those people that change things. She made a lot of stuff for children, and I was one of those who read her books and listened to her songs. Here are a few things she said recently.

My home was an illustrated middle class environment. People sensitive to art, reading, music. That is a huge privilege for a kid. It is as if you inherited a fortune.

Kids do not irritate me, but parents that say "give a kiss to the lady" do. Kids don't like to give kisses. But if I were with 3 children, I'd be comfortable.

Her works are really good. They are about to be republished in Argentina. Unfortunately, you'd have to know Spanish to fully enjoy them.

Quadratic formula fun

Given

  • N = m^2 + r
  • 0 < r < 2m + 1
Then for what values of j does the equation below hold?
  • j^2 + 2jm - r = k^2

Further toy refinement

If N is an odd constant (odd, as in N = 2k+1), when does this happen?

(N - 1) / 2 = i (mod 2i + 1)

Sunday, July 24, 2005

Simpler toys

Given

  • N = m^2 + r
  • 0 < r < 2m + 1
When does this happen?
  • N + i^2 = 0 (mod 2i)
  • 0 < i <= m

Saturday, July 23, 2005

GCD grinding done

By the way, I finished the gcd grinder. The basic idea of how it works fits in a single, reasonable size method. Finally I chunked out all the scheduling mess into a class called SchedulingSetQueue. Fantastic! Now I can do the following:

    14175 factorization refineWithOtherFactor: 105
      < ctrl-p >
    14175 = [3] * [3] * [3] * [3] * [5] * [5] * [7]

All those small factors appeared without even knowing they were there!

News suck, man

This is not democracy, it is not civilization --- it is utter bullshit. And we do nothing about it, which is the most astonishing part.

How about the execs of a French bus company who decide to sue their passengers when they choose to carpool instead of using their service? Twice?

That's news I do not need to read. Back home, remember a corporation called SCO which started a smear campaign against Linux some years ago? Apparently, they knew it was false all along. They do not seem to have cared one bit about it.

From Wikipedia: The Corporation, a documentary exploring the psyche of the corporation, came to the conclusion that if the corporation can be regarded as a person, rather than a legal entity, as it is under United States law, its personality would meet all of the DSM-IV requirements for being a psychopath (such as conning others for profit and recklessness).

Windows Vista --- eat, drink, and be merry

The whole point of Vista is to keep up the sales. Complaining about it being bloated, bad, not innovative, and whatever else is useless --- that criticism does not address its motivations.

In the same way there is obesity, there is consumism. Hence, the cycle of eat+useTheToilet applies to money as well: it's buy+throwInTheGarbage.

If we ever decided to create instead of consume, then the fears so well represented by all those hate articles on the Creative Commons license would become true. Because we might stop working like slaves to have money just so that we can give it to those who have more money than us in return for having whatever is left of our lives occupied with things we don't really need.

As if we had sold our souls to some politically correct Satan.

And just a final word on the "remix culture" argument thrown against the Creative Commons license (essentially, if whatever you do can be used by others, there's no creativity because any "creation" is a remix of something else you didn't create)...

... please explain how movies like The War of the Worlds and Herbie - Fully Loaded are not examples of the remix culture. Please explain how any latest Disney / Pixar movie isn't a previous movie remixed with different characters. Please explain how record company style music is innovative in any sense. Please explain how 100 years of copyright protection is not supporting a remix culture. Please explain how absurd patents are not supporting a remix culture.

The whole point is that in this culture, my friend, our task as the little slaves we are is to work until we drop dead so we can, as much as possible, buy+throwInTheGarbage. A measure of the frenetic pace of this activity is otherwise known as the strength of the economy.

The end result is that just a select few have the chance to stop working like a slave. But as proclaimed and shown on television: economic freedom could happen to you in this land of opportunity! (this offer is provided to you on an "as is" basis, no warranties apply especially that of fitness for a purpose, your mileage may vary, taxes not included, no social security, no medicare, no health insurance, no retirement, working like a slave may be detrimental to your health, excess garbage generation may be detrimental to the environment, to your health and to that of others. For your entertainment only. The terms of this offer may not be the sound advice of anyone or anything. Odds of winning: less than 1 in 1 million. Play responsibly).

Toys

When is it that the following happens?

(h^2 + r) / 2 (m - h) = integer

m, r constants
m + r = 1 (mod 2)
r = h (mod 2)
0 < h < m

Monday, July 18, 2005

On code written under stress

So in my previous post I described how the stress created by having too many names in the execution context is an invitation to constraints just so that it's possible to cope with the situation, and how it would be better to create classes instead.

After some time, I realized something else.

If code is written under such conditions, then clearly it is written from a point of view where self is bound to the developer. Think of the implications for those reading the code later...

In any case, there won't be many occurrences of self being bound to the context where self itself has been written. Rather, self would mean something along the lines of:

  • goto: aNamedRoutine draggingTheseArgumentsAlong: aCollection
  • callFunction: aFunctionName withParenthesisThrownAroundAll: aCollection separatedBy: SyntaxConstants comma
How subtle yet profound is where self is bound when you implement things in a computer...

Saturday, July 16, 2005

On the cost of change and gcd grinding (final)

I find myself stuck with an algoritm that should be easy to implement, but that has denied me the opportunity to write it in beautiful terms.

Given a collection of integers k1...kn greater than 1 and pair-perpendicular to each other, I want to break them into smaller pair-perpendicular factors if a little bird tells me that there is a certain integer m such that for at least one i, gcd(ki, m) > 1.

Well sure enough one could do the easy thing saying g = gcd(ki, m) and then obviously ki = g * ki', thus you'd replace ki with g and ki'. But, observation:

What if g and ki' are not perpendicular?

This happens in the presence of squareful integers (as opposed to square-free integers). For example, it occurs when trying to grind 14175 down using 105 as a factor. First, gcd(14175, 105) = 105, thus you find that 14175 = 105 * 135. You could stop there, but obviously 105 is not perpendicular to 135.

If you want to take things further, you could say well gcd(135, 105) = 15. Now you need to refine both 135 and 105, obtaining 135 = 15 * 9 and 105 = 15 * 7. So now you have obtained new grinders, namely 7 and 9. In short, 14175 = 135 * 105 = 15 * 15 * 9 * 7.

Of course you are still not done. Because if you consider gcd(15, 9) = 3, you find another grinder and after some work you obtain 14175 = 3 * 3 * 3 * 3 * 5 * 5 * 7. Finally, you are done because those factors as a set are pair-perpendicular.

Implementing this is irritating because of the type of recursion necessary. The hackish approach would indicate heavy use of indices, temporary collections and other stuff. But going down that road thinking that refactoring will be able to clean it up later has failed for me. I can't even write a functioning algorithm in hackish terms I can stand.

Most of the problem for me revolves around the scheduling of grinders. In short, I need a queue that also acts like a set and that is thread-safe. I have implemented this kind of thing before in my reference finder, but since the reference finder had a more clear separation between the visitor and traverse processes, each execution context did not have a large number of names being referenced, so modifying the queue of objects scheduled to be visited while the queue was being depleted was not confusing neither it was too problematic from a thread-safe point of view. In short, proper use of an ordered collection was good enough (see below for links) .

However, this gcd grinding process does have a large number of names in each execution context already. Adding the following complexities on top of it makes it unmanageable:
  1. Thread-safe access to the grinder queue
  2. Indexed access to the grinder queue
  3. Avoiding adding repeated grinders to the queue
  4. Avoiding adding a repeated grinder that has already been taken from the queue
  5. Ability to skip a grinder being processed and jumping to the next grinder from any step in the grinding process
Therefore I was stuck. Too many names in an execution context will bring progress down to a halt because your brain has to either a) swap the names in and out of your scratch ram all the time, or b) spend enough time to chunk it all.

But, observation: developers' abilities are usually measured by how much they can brute-force progress in spite of a) without resorting to b). While this rewards efficient thrashing, it also leads to the size of egos being directly proportional to those of scratch rams, both of these being directly proportional to the apparent short-term progress, to the degree to which produced code is an ugly mass of unmaintainable moldy rotten spaghetti, to the increasing disparity of perceived abilities between developers in the same team, to the resulting frictions between team members, to the increasing proportion of brute-forcers in the team as chunkers tend to leave, and to the overall cost of change measured over time.

In particular, the more this happens, the more the code reads like commandments given by a god-like figure. Evidently, the stress caused by the excess of referenced names in the execution context is an invitation to constraining and locking everything down --- as if all objects had to stand perfectly still unless ordered to do something, because otherwise chaos would take over and make brute-forcing impossible.

In my experience, the amount of stress is directly proportional to how much the code gives the impression of somebody shouting orders on top of increasingly louder white noise, while getting closer and closer to a major emotional breakdown.

Since we are not god-like figures with infinitely big scratch rams, chaos eventually takes over and after that any change made to the code leads to the introduction of more bugs than the ones being removed.

Hence, as a humble mere mortal I decided that I had to avoid all that by using classes to compartmentalize the names being referenced at each execution context --- chunking, in other words. The first thing I'm going to chunk out of the mess right now is that scheduling queue. Then things should be easier, and I should be able to write a working gcd grinder within the limits of everybody's scratch ram, namely: 7 ± 2 named chunks.

It is really unfortunate that chunking is usually seen as a sign of weakness (because of the assumed inability to make progress like others by e.g. brute-forcing the situation), and as a sign of time-wasting. Because in general terms, brute-forcing problems past the size of our scratch rams will get us nowhere.

Think of an ant colony: does the queen have to give out precise orders to all ants? In fact, does it have to micromanage by providing control-freak-style orders to every ant while at the same time being aware of what every ant is doing?

Note that the job of the colony is achieved without needing a context in which all the aggregate knowledge of every ant can be referenced using some labeling convention. The colony's self-awareness does not have to be omnipresent. Same thing with your own body which is supporting your act of reading this.

There is not even one example of a complex system needing an omnipresent self-awareness context to evolve. Hence, there is no need neither excuse for brute-forcing code into existence.

* Examine the reference finder's breadth first traverse implementation by downloading the source code from here (and download RAR if you don't have it already), or by going to the Cincom public repository and loading packages Distinctions and ReferenceFinder.

Thursday, July 14, 2005

Unexpected use of exceptions

Typically, when you use an algorithm to determine if a number has strict divisors, you use the algorithm to both factor into primes and to determine primality (no strict divisors found).

But how irritating it is to write an algorithm that can do both efficiently! Because if you are just looking to determine primality, any divisor found already gives you the answer --- so you'd need a check to see if you should stop. But when you want to find all factors, then you don't want to check because it's unnecessary.

And yes you can subclass, and you can do all sorts of complicated stuff, or you can raise a notification when you find a factor! Since notifications' default event handler proceeds, nothing bad happens if nobody is listening. Thus,

    Integer>>isPrime

      ^[self refinedFactorization isDefinitelyPrime]
        on: ComposedIntegerFoundEvent
        do: [:ex | ex return: false]

Wow... so much nicer!

Observation: usage of when:send:to: within the same process amounts to raising a notification. Think about the implications...

Sunday, July 10, 2005

Regarding Dennis

As an expression of nature, hurricane-like storms are extraordinarily beautiful. When we see them in other planets, we seem unable to help thinking they are gorgeous.

There's not much more to say about the damage hurricane-like storms cause on this planet's landmass, though. I just wish we didn't get in their way.

Did you know the point of any storm, as far as nature is concerned, is to cool down the surface?

Friday, July 01, 2005

Near-optimum Nysqually strategy

Do you know the game Nysqually? If you do, here's a near-optimum playing strategy.

  • "Reduce playing rectangle"
  • Get a histogram of all types of tiles.
  • Pick a side of the playing rectangle, count its length.
  • Pick a combination of groups of 3, 4 or 5 tiles such that the sum of their sizes is equal to the length of the side of the playing rectangle you chose. Make sure that you never choose in a way that leaves 1 or 2 tiles of the same type (since otherwise you won't be able to take them out).
  • Assuming it's possible to shuffle tiles around without making a mistake or inadvertent tile deletion, move them in such a way that the side of the playing rectangle you chose goes away.
Execute until the whole playing rectangle goes away. Works best by interleaving horizontal / vertical sides in your order. Therefore, after every 2 sides, you always end up with a square.

For easier playing, favor tile choices so that you tend to constraint tile counts to multiples of 3 when you are about to have a rectangle whose sides' lengths are multiples of 3. When you reach that situation, then you can take out the whole thing in columns / rows of 3 and you win.

When you have to take all tiles out starting with a 12x12 playing rectangle, no strategy will almost certainly make you lose. The strategy above, however, makes the game quite boring since it appears you cannot lose --- but I have not proven that, hence the near-optimum claim :).

Number 10 is mine, mine and only mine

So I come back from work and I read these two emails saying that some dude I know from Spain has a) tied my world champion record in Hat Trick Hero (56 goal difference over 7 games), and b) has posted a higher record in Euro Football Champ 92 (45 goal difference over 7 games vs 42).

I was furious. So I decided this situation wouldn't last 1 day. It didn't even last 10 hours.

Euro Football Champ 92 now has a record of 49 goal difference with zero against.

Football Champ now has a record of 58 goal difference with 1 against (59-1, improving my previous recording of 53 goal difference, 56-3).

Hat Trick Hero now has a record of 66 goal difference with zero against.

Good-bye extraneous results!

Sunday, June 26, 2005

Gorgeous method

If you are into primes, then you know of the Mersenne numbers: 2^n - 1. Here's my implementation of Integer>>mersenne, the answer of which is the receiver-th Mersenne number:

    mersenne

      ^self ones

Isn't that beautiful?

Friday, June 24, 2005

ADPCM beautiful solutions

Now it's the turn to rewrite my ADPCM codecs.

Couple things... these are lossless ADPCM codecs geared towards audio. My previous implementation accepted 8/16 bit, 1-256 independent channel inputs. It had plug-and-play delta strategies, and there were several more or less hard coded coding strategies.

Note to self: without knowing it, I experimented with Rice coding. What it does is to write 1 bit to tell if each delta you write has overflowed or not. But if your delta modeler is good, that means that most of the time there's no overflow, which means that most of the time you're penalizing each delta with 1 bit. B-b-b-b-bad to the bone!

It's much better to reserve the top and bottom values of your allowed delta interval to signal upper or lower overflow, and follow that with a special coding for the rest using the fact that you already know the minimum size of the overflow. Being able to represent 2^n - 2 values instead of 2^n is a much less expensive proposition than using 2^(n+1) values to represent 2^n values.

So how to reproduce all these features without writing a ton of messy code... first, I chose a 4 stage approach to coding/decoding. Coding consists of calculating a delta according to the delta strategy, then updating the coding strategy with the current delta strategy configuration, then encoding the delta using the coding strategy, followed by updating the delta strategy with the efficiency of the coding (this gives feedback to the delta strategy so it can avoid being too aggressive or too conservative). This means that an ADPCM codec has two strategies, which instantly means they are serializable objects which are written to disk via serializable parameters.

Decoding consists of reading a delta according to the coding strategy, calculating the original value and outputting to the destination, followed by updating the delta strategy with the observed coding efficiency. So far, so good.

To implement multiple channels, I decided to implement a mux / demux codec with a list of scheduled codecs (the list also becomes a parameter, its elements are serializable). To implement different bit depths, I decided to implement a stream splitter codec.

So a full ADPCM codec is a complex codec: stream splitter >> mux / demux >> basicAdpcm {with strategies}. Decoding is accomplished by directing data from the adpcm through the mux / demux through the splitter.

I like this stuff very very much.

Wednesday, June 22, 2005

Coding stuff news

My new version of codecs have seen their first successful code/decode/compare run. I'm very happy with this!

So far, I just have an identity codec that does nothing. The trouble up to this point was how to envelope the coded symbols in a self-explanatory way so that a generic decoder would be able to bootstrap the coder. Now I have a serialization mechanism capable of serializing cascaded codecs with all their parameters, in a definitely declarative approach (the serialization is not a file out that is later dependent on the runtime).

It's not a tough problem. But I wanted an elegant solution.

There are serializable objects. Codecs are serializable. Their parameters are SerializableParamater objects.

A serializable object typically starts its serialization by writing its class name in ascii-z format to the destination stream.

A serializable parameter object refers to an getter/setter message pair understood by its parent codec. Its serialization is its class name in ascii-z format, followed by the getter selector in ascii-z format, followed by the serialization of the referenced value.

If the referenced value is a codec, then its serialization is the serialization of its class name plus the serialization of all its parameters. Thus, when a codec envelope is written to disk, the write operation is cascaded through the first codec and then through its serializable parameters.

Different subclasses of SerializableParameter determine what is written/read at serialization/reification time.

The read order is the same as the write order. Hence, it's not necessary to keep track of the current reification context since objects are always reified in the implicit context --- which always happens to be correct one.

Envelope handling pollution is avoided by having EnvelopedCoded deal with kicking off/boostrapping the envelope writing/reading process.

The first successful serialization was:

EnvelopedCoded[Z]CodecParameter[Z]codec[Z]IdentityCodec[Z]

Sunday, June 12, 2005

More Maradona stuff

I am happy because it's comparatively easy to find rare footage with the Internet.

Maradona videos are some of those available gems. It's great to watch his stuff and listen to radio broadcasts of his plays. Nobody else was or is as good. Incredible, amazing, mesmerizing, forcing you into disbelief, into a there-is-no-way-I-just-saw-that feeling that electrifies your skin.

Taking a step with his left foot while running, barely pushing forward and up a bouncing ball before planting the same foot on the ground, thus making a defender slide by in front of him getting just air, and then taking a little jump on the same foot he just planted to go over the rest of the sliding defender. A tenth of a second movement executed just like that, while keeping another defender still behind him.

If you know Maradona and you watched him play, you know what I am talking about. And then there's training session footage. The ball, still on his left shoe, is kicked upwards and towards his back. He turns around and with the same foot catches the falling ball in the same spot it was before, with the exact opposite force so that without any other movement the ball is laying perfectly still on his shoe --- just like before.

Then you see the amazing variety of juggling with his thighs, shoulders, chest, both legs, heels, keeping the ball still on his head while walking or jumping... then the same with oranges, lemons, pebbles, balls of crushed paper, half-filled bottles of mineral water, tennis balls, golf balls, and golf balls with dress shoes...

Then you watch his plays. Going by a goalkeeper and confusing a panicked defender so much that he slams himself with the goal post between the legs as Maradona simply pushes the ball to score. More defenders ripping patches of grass up in the air trying to catch up, trying not to break their bodies as they stop and stretch, but always too late. Goalkeepers staring at the ball in frustration, disbelief and shame --- scoring from the middle circle right after a kick off, or from 40 yards out, or after running half of the field and finding the keeper too far removed from the goal.

Then his most famous stuff in the match against England in the 1986 world cup. 5 minutes in the second half, game tied 0-0, he gets rid of 4 defenders, then passes to a team mate, a defender kicks the ball up in the air, the goalkeeper runs to get the ball in the box, but Maradona is there. And somehow he manages to jump higher than the goalkeeper and pushes the ball into the goal to score. A beautiful play because of his dribbling, because of how he managed to be in the right place at the right time to beat the goalkeeper's jump. You realize how much skill it takes to do that.

But there is a catch --- the referees did not see that he did that with his left hand, not his head. It's hard to see even on TV.

The english players are furious with the referees, and with good reason. This time however, there's much, much more to that. Because England stole the Falkland Islands from Argentina on 1830 or so. In a complicated series of twisted events, and after 150 years of diplomatic complaints tainted with duplicity, the crazy generals of a US-supported dictatorship who had "disappeared" 30,000 people or so decided to do something patriotic for a change and kicked the English out in 1982. Argentina lost the following war against England with the help of US weapons. An extremely humiliating affair.

A couple things are evident. First, crazy people desperate to stay in power start wars even when they cannot win. Second, poor countries will always take cheap shots from rich countries which stole everything from them to begin with --- because those in the poor countries are always too few, always too poor, always too alone, isolated and helpless as their rights are stripped away from them over and over again, since the beginning and until the end of time.

So with emotions running so high, you are given an illegitimate hand goal against those "thieves" from that other rich country. Even though you know most people from England had nothing to do with the Falklands being stolen, you just conclude "too bad". You won't prosecute the offender this time because the history you know and the emotions you feel don't let you. And at least once, something unfair went your way. Even if it is frivolous, you owe Maradona the satisfaction of knowing that in some other country, they are the ones who are incensed and helpless at the unfairness of the situation right now.

So you'd think enough is enough, and you are satisfied with this alone. Regardless of the fact that you've been robbed so much more than a hand goal, you feel the sweet taste of revenge --- even if you know that after all is said and done it's insignificant.

But Maradona is so much more than a smart ass cheat. He is a one-of-a-kind artist as well. And so that there's further humiliation, he does something even more amazing not even five minutes later. He legally accomplishes what nobody else has ever done.

Taking the ball in his own field, he turns, steps on the ball, and leaves 2 opponents behind. One of them is left in such stupor that he does not run after him. As if taking some ballet step, Maradona controls the ball and with a quick change of direction he goes between 2 other opponents and now the count is 4. The sweeper waits for him at the edge of the penalty area, but Maradona makes him number 5, leaving him as if attached to the ground after a quick direction change and a little jump. The goalkeeper comes trying to keep him from scoring, but he becomes number 6 as he is left spread on the grass after a deliciously subtle shot fake.

Then Maradona slows down ever so slightly so that he's back on his left footing, so he can score with his magic left foot. And even after giving this advantage to the last defender who fouls him from behind, not even this 7th challenge keeps the greatest goal in soccer history from happening as a final defender gets to the goal too late to contest the soft shot.

And now you realize you owe Maradona way too much, and that there is no way in which you can possibly pay him back. First he cheated, stealing from them in their faces, yet forcing you to ignore his infraction. On top of that he has done something that through your eyes and skin has become imprinted on your mind forever, that made millions of soccer-savvy fans drop tears, crying in disbelief and amazement at his greatest artistic expression. Because in those ten magic seconds, Maradona's number 10 became further associated with the best.

From us, those who appreciate art in soccer: thank you so very much Diego. Thank you for your soccer, for the joy the unique beauty with which you play gives us, and especially for this Argentina 2 - England 0.

Tuesday, June 07, 2005

Success tax

The basic idea of writing a computer program is replicating communication relationships between distinctions in the computer.

Then why would you do anything to tax communication between objects? In this sense, Java is a bureaucracy, as below:

An administrative system in which the need or inclination to follow rigid or complex procedures impedes effective action: innovative ideas that get bogged down in red tape and bureaucracy.

Let's respect each other, shall we?

Why in hell should I pay $800 to go to the emergency room because e.g. I have blood in my urine, just because I only have Medicare?

We think we have it so good... and as many other third-world countries, we just garbage collect people who can't increase the disparity in the distribution of wealth.

But all is great as long as it doesn't happen personally to us, right? Such bullshit.

Monday, May 30, 2005

Doug Engelbart's demo

I watched all of Doug's 1968 demo today. I cannot help feeling we believe progress was made just because some eye candy has changed. Even today, some of the things Doug et al showed back then are still sci-fi for most of us. Sigh...

Painting cascaded codecs

One of the projects I'm working on is my compression stuff. I've neglected it long enough already...

Something I am working on right now is the ability to cascade codecs. The main problem is that I do not want 500mb to be processed by a codec, then the resulting 400mb by another, then the resulting 300mb by another and so on.

This is problematic because codecs will typically output bits instead of bytes so you can't simply have a byte-sized symbol propagate through the stack just like that --- you'd have to implement tons of boundary checks, and ifTrue:/ifFalse: will kill your performance on top of being ugly. So what to do...

The other day I found a very nice solution. Typically, a simple codec will have source/destination streams. Initially, complex codecs will look that way. When you add another simple codec to the complex codec, it goes from

    sourceStream
    codec 1
    destinationStream

to

    sourceStream
    codec 1
    forwardingBuffer 1
    codec 2
    destinationStream

Forwarding buffers are growth-limited "write streams". Upon reaching their size limit, they will #flush by sending #runOnSourceStream: self to the recipient and then reinitialize themselves. Thus, flushing is performed only when it's necessary.

Flushing on demand to finalize the destination stream is accomplished by cascading the message #flush across codecs, forwarding buffers, and streams. No special case logic needed.

Extra points: what's the instance creation method?

    ForwardingBuffer
      store: anInteger
      thenForwardTo: aRecipient

Beautiful, just beautiful!

Sunday, May 29, 2005

Maradona

So how come ten is a good number and I haven't written a word about soccer star Diego Maradona yet?

Maradona was the best in a time when soccer was much more geared towards destruction of play than ever before. And he did the most astonishing things while having fun. Most importantly, he is an artist. I find great pleasure in watching him perform.

But, unlike Pelé, Maradona is quite unheard of here in the US. Maybe it's because of his trouble with addictions --- but wait, isn't there a lot of noise about widespread doping in US flagship sports, haven't popular US track and field athletes been banned for life? And the reactions look too much like CYA and business as usual. So Maradona's lack of popularity can't be a chemical problem and there has to be another reason.

Maradona was born extremely poor. His father worked 14 hours a day to feed a numerous family. He had something like 8 brothers and they all shared one of the two bedrooms in his house. People in his neighborhood often went without eating. There was only one drinkable water faucet in his block.

From this place he was thrown on the throne of soccer by means of a powerful kick in the rear end, as he says. With his background, he had to deal with his country believing they were the best because of him, and with being the best in Europe with the expectation that he'd behave with Victorian gentleman manners. Somehow that didn't quite work.

For example, when he met John Paul II, the pope gave rosaries to his relatives. When it was Maradona's turn, the pope told him he had "something special for him" --- and gave him another rosary. When Maradona checked, it was identical to the other rosaries. So he went and asked the pope "this one is the same as the others, how is mine special?". No answer.

Maradona also saw all the wealth in the Vatican, and how its residents don't go through hardship. But especially after being criticized about spending too much money in his own catholic wedding party, he answered back commenting on all the money the Vatican would spend on Africa trips where the pope would kiss the ground, avoid leaving any money for kids starving to death, and still keep the gold hanging from the roof back at home.

Maradona is very direct and he usually expresses himself without political correctness, which is just another euphemism for being hypocritical and exhibiting amoral behavior.

Since Maradona is a human being, he has the right to express himself in any way he wants. It's amazing that I have to write the previous sentence, as if it wasn't obvious. But it may make you realize how much public figures don't do that because they'd rather shy away from the enormous responsibility that comes with standing for your beliefs.

Even though he was not specifically prepared to do so, Maradona had the presence of mind to take a huge responsibility on his shoulders. In my opinion, he outperformed the vast majority of us at that.

I wish we had more courage to ask difficult questions. But in this country that kind of behavior is not fashionable. That is probably why Maradona, like so many others, is unheard of in the US.

Saturday, May 28, 2005

Time to paint

I realized how much of what I do when I write stuff for myself is like painting, or composing music... creating something in an artful way.

I don't understand why people would prefer consuming instead of creating. Because see, instead of creating new things, that darned grass is always tall and you need money to buy a tractor to cut those 50 acres of grass you bought on credit. Your 5000 sq ft home is not full enough of things craving your attention, so you need to spend your weekends shopping and doing forklift work. Cars are not enough, SUVs are not enough, you need to commute using a moving truck just so you can carry 100 of your coworkers plus all their shopping plus the new team equipment for that soccer match...

I don't buy any of that. I want to create. And today is an excellent day to implement Life in a way I've never seen it done before.

Life is short --- time to paint.

Gershon Kingsley

Electronic music... ahhh... my everyday programming music. And this guy, Gershon... most of his songs last under 3 minutes and they are the appropriate length. What nice music!

This world is ridiculous

"Happy memorial day" - Lockheed Martin, world's largest weapons manufacturer.

Sunday, May 15, 2005

Couple things on deep packet inspection

Today I read a bunch of stuff that I found extremely disturbing. Deep packet inspection is helping ISPs turn the Internet into a dumb-terminal world. In short, you pay tons of money to have the "privilege" to pay more tons of money to get "stuff". And that's it. Nothing can be shared, and nothing is for free in this "golden" environment. Plus you get zero privacy and all your communications are spied on.

What really pissed me off was that a company offering deep packet inspection hardware targeted free VoIP software Skype in its marketing literature. In other words, bandwidth spent on Skype is a lost revenue opportunity. So what you as an ISP should do would be to purchase deep packet inspection hardware to block Skype traffic, then turn around and sell VoIP services to your customers at a higher price.

This is utter control freak bullshit and I am at a loss to explain the indifferent behavior of conslamers (a mixture of conned consumer slaves).

Clearly the problem is the greedy ISP serving big corporation interests to turn your life into a Truman Show. So all of that needs to be removed from the picture. And before you say "where am I going to get my email address and stuff then?"... remember how the Internet started? Few machines connected via TCP/IP, a resilient transport protocol? New York is a huge city with over 10 million people in it. There should be at least 1 million computers here. And there's this thing called wireless networking.

If you didn't have to deal with your ISP, wouldn't you encapsulate your home network, provide an interface to your sorroundings, and let TCP/IP do the dirty work of properly routing traffic in the same way your ISP does? Sure it may not be as efficient at first, but it would be free. And for privacy purposes, you could simply run something like Freenet on top of this free network so you get your privacy back in a medium that would be harder to contaminate with unsustainable dreams of endless exponentially growing profits.

P2P applications such as VoIP could easily run on this, without a single telemarketing call because of RSA keys.

An issue would be how to connect large geographically disjoint parts of the network. I think ISPs would like to have any traffic whatsoever by then. But maybe that could also be unnecessary --- you could run your wireless hubs in the car, in the truck, in the RV park, in your privately owned land thanks to donations signed and processed thanks again to PGP, etc...

So you'd keep your ISP for accessing dumb-terminal stuff. And for the really interesting stuff, just go wireless and remove the middlemen completely from the picture.

There has to be something wrong with this, because if it could work I am sure it would have been done already. Maybe it exists, and because I am still connected to the dumb-terminal internet I do not know about it! If it isn't, I wonder what is the limiting factor, besides bandwidth and FUD.

I hope that somehow we will learn to dismiss our conditioned fear reflexes soon. I think life is too short and extraordinary to waste on a quest to amass a pile of papers other people judge as valuable on grounds as weak as they get.