Wednesday, 12 November 2008

The Importance of Transparency

I know some people who read this can't be arsed with the technical posts, but I do use this blog to tell the world, including my friends, what I'm up to, so please bear with me :)

People > Data > Code > Hardware

That sequence represents two things. Firstly, if those are treated as arrows, it shows how computer programs are generally used. A person inputs some data and the code does something with the data by running on the hardware. Another way of looking at it is as an inequality. People are more than data, data are more than code and code is more than hardware.

Hardware is a lump of plastic, silicon, germanium, steel, etc. It only exists to run code, therefore code > hardware.

Code only exists to manipulate data, whether those data are numbers in a calculation, images to be displayed, music to be played, messages to be sent, etc. Therefore data > code.

Data is only kept around because it is of use to people. Despite our best efforts, hardware and software cannot appreciate the humour of a LOLCAT image. Therefore people > data.

This relationship can be seen in many areas. If I have the most awesome server ever, nobody gives a crap if my Web site is crap. Google's search engine started life on incredibly underpowered, unreliable hardware, but nobody noticed because the code was redundant and reliable. Mugshot.org may be coded better than Myspace.com, but nobody uses Mugshot and there are far more data in Myspace.

Hardware doesn't matter, so as much as possible should be as cross-platform as possible. I have Linux running on my desktops, my laptops and my 'phone. Windows will only run on x86 and x86-64 machines, which means no phones, no PDAs and very few embedded devices like set-top boxes and games consoles. If code is cross-platform then users don't need to give a shit about hardware, which makes life a hell of a lot easier.

Code doesn't matter as much as data, so as much as possible should be in standardised, implementable, documented formats. The spreadsheets I write in OpenOffice.org also work fine in Gnumeric and have live copies saved on Google Spreadsheets. Spreadsheets made in Microsoft Office 2007 can only be opened in Microsoft Office 2007, since everyone else's attempts at compatibility are flawed. If data is openly standardised then users don't need to give a shit about software, which makes life a hell of a lot easier.

This just leaves people and data, which are the only things that are important (code and hardware are just tools used by people to manipulate data).

Using the examples above, I can save a spreadsheet on my desktop and access it from anywhere in the world via Google Spreadsheets in the browser on my phone. The proprietary alternative is to only be able to use Microsoft Office 2007, which requires Microsoft Windows, which requires x86/64 hardware. A very cosy position to be in for Microsoft, but for the vast majority of the world who are not Microsoft employees, why give up so much? This isn't just a feature argument either, since Microsoft could make a browser-based spreadsheet system. The argument is WHY DO I HAVE TO WAIT FOR MICROSOFT? If you hand someone the keys to your data, you should expect to be taken for a very long ride, at the end of which you might not even have that data any more.

The same goes for Facebook and other proprietary applications. (Free Software doesn't always use standard formats, but the formats are at least documented to some small degree in the code. Proprietary apps give you no code.)

END COMMUNICATION

Tuesday, 28 October 2008

Medibuntu :P

Just read that the one of the GNU project's Free Software desktops, Gnome, is being used in a new machine which uses flesh elasticity to find breast cancer. That's an awesome reason to hack on Free Software, since its impact on the world literally IS saving lives :)

The company, SuperSonicImagine, aren't just (ab)using Free Software because it's royalty free, like many companies do. They use the Cairo vector imaging library, amongst others, in their own software and contribute patches back upstream so everyone benefits from their improvements :)

Also, if you're in a recent version of KDE 4, try opening the Akregator feed reader, opening some Web pages in its embedded Konqueror part, open Epiphany, then drag a tab from Akregator over the panel entry for Epiphany until it switches, then drop the tab into epiphany and watch it open. Pretty nice integration going on :D

Saturday, 18 October 2008

Amarok and roll

Just installed Amarok 2 via the Neon nightly build service (for Kubuntu, but still works on Debian Unstable :D ). I have to say, it is absolutely teh awesome! It really shows what KDE 4's "pillars" can do when used in combination.

Don't get me wrong, I've used Amarok 2 before. However, the first time I tried it it kept crashing and didn't have much functionality. My attempts since have been greeted with a feature-packed player, which couldn't actually add anything to the playlist other than Internet radio streams :( With the nightlies it seems to be rocking full steam ahead, and doesn't take up too much space either :)

The only problem currently is that the Qt it uses doesn't seem to have antialiasing enabled for text, but that's a very minor issue.

Now to get Nepomuk to run without making my machine crawl. Perhaps the Java backend would make it work faster?

Thursday, 9 October 2008

I CAN HAZ OPENMOKO?

Firstly, thank you to everyone who gave me mobile phone advice. I've probably disregarded all of you with my eventual purchase, an OpenMoko Freerunner :D

The Freerunner is pretty nice, except of course that every piece of software on it is an utter pile of unusable fail. However, I paid for the phone and not the software, so I'm going to try and make some decent apps for it.

At the moment I'm getting responsibilities piling up though :( I've got homework assignments, Free Software Society organisation, Computer Science Society collaboration to manage, Access Space, ShefLUG and Manchester Free Software collaboration to manage, Sun Microsystems relations, out-of-hours programming for my course to catch up on, relationships, general society stuff like RockSoc and things, Free Software programming (some of my projects are in dire need of some time, which I simply don't have), I have a pile of books which I want to read, the constant barrage of email, XMPP and RSS, going to the gym (which I still haven't done), sorting water stuff, sorting rent and contracts, sorting legal bollocks, sorting TV license, sorting the electoral roll and to make matters worse I'm rather ill, resulting in painful sneezing and coughing fits (I've even had to take some days off, which means missing lectures, which means catching up :( ).

I think the way to tackle this is delegation. The Free Software Society takes up a lot of my time, but is also full of awesome people who gladly help out. By getting these people to help out, like Arthur organising the socials, it will result in a lot more free time in which to do the other stuff.

Still, I've only got the one life, so why waste it? It does annoy me though, when I spend two days attempting some Physics homework and make absolutely no progress :'( That's at least 12 hours I could have used to get something else out of the way, but unfortunately I fail at Physics too much. The solution is, of course, to work even longer on it. Bugger.

Ah well, I don't have time to blog away so I'll wrap this up now.

By the way, PubSubClient is now hosted on GitHub rather than Gitorious due to a request in my blog comments.

Until next time!

Sunday, 5 October 2008

On People

People are intelligent, sentient beings.

If you try to deny the intelligence or sentience of any human being then I suggest you give me an incredibly wide berth.

Sentience, in my mind, is the only thing of any value in the entire Universe. I don't give a crap how well you can recite Pi, or tell funny jokes, or how awesome you are at anything. If sentience is at stake, I'd bet you and I together in an effort to save it.

For those who belittle the sentience of others, I severely wish you're being ironic, or you may meet an untimely end at my hands. Yes, that is a death threat, but laws don't mean anything compared to the continuation of sentience, so I would gladly accept whatever temporal punishment may be wrought against me in a fascism such as the UK if it means an ontherwise unobtainable continuation, or possible continuation, of sentience.

Some people are crying out for bricks to reshape their skulls. However, such sentient beings are part of the solution, rather than the problem (for the time being), so the best course of action is to disassociate oneself with acts one finds disagreeable.

I hope that offers some useful philosophy for someone, as it is useful to me.

Sunday, 21 September 2008

Brains are Object Oriented

Following on somewhat from my browser UI rant, an important thing to realise when designing interfaces is that the brain is used to realistic, real-world things. It gets confused by magic, tries to reject Quantum Physics, and so on. In other words, the brain is Object Oriented.

To be Object Oriented simply means that there are a bunch of "things" which can each do stuff and each have stuff. For example, a ball is a thing. It can bounce and roll, and it has size and colour. Being Object Oriented means breaking apart some things which seem to logically go together, like verbs (or in the case of a computer program, functions). However, with really huge projects it is clear that separating like from like and instead combining things based on context (into an object) is less confusing, since this is more understandable to our brains.

With user interfaces there is magic and quantum "spooky action at a distance" which the brain does not like. This is where changing one thing affects another. For instance, the up and down arrows on a scroll bar. These arrows are not connected to the slider, so how do they manage to move it up and down? By magic. How often do you drag the slider on a scroll bar rather than press then buttons? This is because the brain thinks of moving down as a method of the slider, rather than the arrow being a function which moves the slider. For this reason I propose that arrows be taken off scrollbars (some themes to it already).

Another problem is toolbars. They contain precisely that, tools. In the real world when we want to join two pieces of wood together we think about putting a nail through them. In order to do this we get a hammer, but that's just an implementation detail. If we could push the nail in then we would do. Thus toolbars are bad because they are a kind of magic. They contain functions, ie. they perform actions, like save(document). However, the interface *should* provide methods, ie. the documents should be able to perform the action, like document.save().

A key thing to realise is that your application *is* the GUI. Therefore your GUI should be Object Oriented as far as possible, and should itself act like an object. Developers often think of their code as being the heart of an application, with the graphics as a glossy veneer on top. Users, however, think of the graphics as being the heart of an application, and the code just sticks it together. There are far more users going to be using your software than developers (you should hope :P ) so keeping this in mind is important.

Video or it didn't happen

I've made a screencast to show some functionality that PubSubClient provides. Doesn't play in SWFDec at the moment, so you might have to use something like this. I accept Google Video's terms of use, but not YouTube's.

The video shows the Browser test application being used to find the nodes on a server and their sub-nodes, add and delete nodes, and add and remove node owners. I've been busy tonight, so that the display issues which cause me to close and reopen windows to show updates are fixed, I've got the Owners functionality working for Publishers and Outcasts too, and I'm mulling over a Subscriptions tab in the node properties (awkward since it may involve message stanzas, not just iq stanzas), generating the contents of the Metadata tab dynamically based on what's defined and changing the access model and other configuration options (awkward since it involves requesting, submitting and cancelling XMPP forms).

For now though, I'm getting tired so it'll have to wait.

Saturday, 20 September 2008

PubSubClient has a home

I set up a Gitorious repository for PubSubClient a while ago, but couldn't actually put any code in there because my SSH key wasn't working with the pass phrase I remembered (ie. I forgot it :P ). Now I've revoked that key and added a new one, so I can upload again :D You can find the complete source there. I've also created a Sourceforge project for it, so that I can give it a free website (Google Code hosting doesn't allow GNU Affero GPL licensed software :( ). The project is still pending approval, but I feel confident that it'll get approved since it is pretty novel.

The repository contains the library in its current form, as well as the ever-improving "browser" test, the currently-stalled "reader" test and the not-really-started-yet "writer" test. To use the library itself all you need is lxml and xmpppy, then just stick the "pubsubclient.py" file in the same folder as your application and you're ready to go. The tests make use of some other libraries, since I wanted to demonstrate that it is easy to integrate into any existing toolkit. To use the browser you'll need pygtk and kiwi, the reader needs pygtk, kiwi and webkit, whilst the writer isn't worth bothering with ATM.

Have fun :D

PS: In other news Jabber Inc. have been bought by Cisco, which is pretty cool. There's some confusion though so remember, "Jabber" is just a nickname for the messaging protocol. Its actual name is XMPP (eXtensible Messaging and Presence Protocol), since it may have originally been created by Jabber Inc. a decade ago, but for the past few years it's been looked after by the XMPP Standards Foundation. Jabber Inc. (who own jabber.com but NOT jabber.org) can of course add to and change the standard, but they must follow the same proccess as Google, Microsoft, Facebook or anyone else who wants to do so. In other words: This is a Good Thing(TM) since XMPP is being adopted heavily by a huge firm like Cisco (who previously used the SIMPLE messaging standard, which is anything but), and the XMPP standards are still being kept safe by XSF as they were already. Rock on!

Spot The Difference

Here's Google Chrome on Fedora:

Here's Firefox with a "Google Chrome theme" on Fedora:
What's the WTF-were-you-thinking mistake? Can't figure it out? How about if I tell you it also applies to the default Firefox theme on OSX:

But not in the default theme for Windows:

Or Ubuntu:

Still can't see? How about an analogy:

Still don't understand? Let me spell it out:
Chrome gets this (as does Opera, but I don't give a shit because their browser is still proprietary), but this supposedly-Chrome theme (as well as the OSX theme) have made the bad UI of Firefox EVEN WORSE. The default Windows and Gnome themes have tabbed pages, with some unknown, magical force changing the address bar when switching tabs. This is BAD since it requires either reverse-engineering, or study of the code, to find out the behaviour. However, with those 'upside-down' tab themes the tabs are now controlling the address bar, with an unknown, magical force changing the page content accordingly. This is EVEN WORSE since it is the page contents that people actually care about! Hype Firefox all you want, but I prefer looking at Web SITES to looking at Web BROWSERS.

In short: EPIC FAIL

Friday, 19 September 2008

Wanna Be A Programmer?

Computers are all around us, and our lives are to a large extent controlled by them. Your bank balance, your phone calls, your email, messaging and Web browsing, games, wages and bills, music, video, literature and image production, trains, planes and cars, it's all controlled by computers.

Software is the stuff which controls computers, and therefore our lives. So who controls the software? Well, if you deal with proprietary software then it's usually controlled by some foreign corporation trying to wring money out of you. If you deal with Free Software then the control is still in the hands of the developers, however YOU can choose the developers. If you don't like what the original developers are doing then you have every right to fork the software and get some other developers to work on it and take it in the direction you want. Often this involves paying the developers a wage, but there is also another option: YOU can develop it yourself! The only skill you need is computer programming, which is a skill that can even be picked up by young children, as long as they have some exposure to it.

So how can YOU learn to program? Well if you're a complete beginner I would follow these steps:

1) Forget EVERYTHING you think you know about programming.

It's most likely bollocks, perpetuated by the media's obsession with stereotypes and putting flashing, unintelligible crap all over any computer monitor that appears on screen, and the proprietary software companys' desire to look like the crap they serve is somehow magical and unequalled. It's not. Programming is easy. It has to be, because computers are so unfathomably stupid. Just because Turner is able to create beautiful seascapes doesn't mean that a two year old can't scribble with a crayon, in the same way that simple programs are easy despite huge ones being hard.

2) Learn to think like a programmer.

2.1)

A nice way to get the hang of this is to play games like this one (NOTE: You may need to install SWFDec to play that). Light Bot features a very simple programming language to control a robot, it is easy to use since each command is an icon and the program (called "main method") is a grid which you drop the icons into then press Go (the program runs like English text, ie. starting at the top left and going across, then going down a row starting at the left and so on). The nice part about it is that the program can only be a maximum of 12 commands long, which is not enough to complete some levels, but there are two functions (reusable blocks of code) available to make repetitive things take less space.

2.2)

After you've completed that game a few times (there are 12 levels) you can go on to Guido van Robot (this may be in your package manager already), which is very similar to the Light Bot game but uses typed commands rather than movable icons, allowing you to make much longer programs. Even without Light Bot's restricted space it is still important to use functions like you've learned, as they are the best way to make large programs more manageble and less confusing. Transitioning from Light Bot to Guido van Robot is an important skill, since it makes sure you know the principles of programming (commands, functions, repetition, etc.) and how to use them, rather than just learning one specific programming language.

2.3)

After you've mastered Guido van Robot and can comfortably tackle any problem without difficulty or confusion, it's time to move on to something a bit less rigid in structure, like the old favourite Logo (there are a number of different Logo programs out there, you can try XLogo and UCBLogo). Logo is a programming language in its own right, but is often used with the "turtle" graphics system, a very similar concept to controlling the Light Bot and Guido van Robot (it can actually be used to control a real robot, but I'm assuming you don't have one :P ). The turtle can be moved to any pixel on the screen by giving it commands like FORWARD, RIGHT 90, and so on, plus it can draw a trail as it goes, allowing you to draw images. Taking advantage of the incredible speed of modern hardware you can make animations, by drawing lines, erasing them, then drawing them again in a slightly different way. Give yourself some problems to tackle, such as drawing a circle, making a function which can draw a circle to a given size (this uses the concept of "arguments"), drawing a spiral (this uses the concept of "recursion"), a stick man, an animation of a bouncing ball, making the ball squish slightly when it bounces, a windmill with turning sails and a growing vine. All of those are possible with a little thought, but *remember to use functions* and leave yourself *a lot* of comments throughout your code to remind yourself wha each bit does. When you can make those given examples without much thought you've mastered a lot of the art of programming. Well done :D

2.4)

There are many books out there which can help you learn to program. For instance How to Think Like A Computer Scientist. Googling for guides and tutorials on your language of choice is usually helpful too.

3) Learn one powerful programming language

Logo is a nice little language, but your programs can't really talk to anything else. This requires software called "libraries", and for libraries which are not written in your language of choice you will need a "binding" to access them. I don't know of ANY libraries which are written in Logo, or any bindings to let Logo access libraries written in other languages. This means you'll have to move on to something more powerful :( Since you now know Logo, which is a simplified form of the LISP language, it would be sensible to learn another simple, LISP-like language. I would recommend Scheme, which is a very powerful type of LISP, without a lot of the unneeded crap which exists in LISP. Guile is a very good Free Software Scheme system, which has bindings for many libraries, therefore I recommend your next step is to dive in and learn Scheme using Guile. It is very similar to Logo, except that every command is enclosed in brackets. If you want to make real graphical applications, there are bindings for the Gnome desktop, and even the new Clutter 3D graphics library. Now that you know Scheme you can do anything. Although some things are harder than others...

4) Learn the Object Oriented paradigm

In the same way that functions are an INCREDIBLY useful tool to make programs short, readable, understandable and maintainable, there are a few different progamming 'paradigms'; ways to structure and think about your programs and how they work, which will make your life easier and avoid getting headaches.

The first paradigm you used was "procedural programming", where the code describes the procedure to follow, "do this, then do that", but there are many more, some of which you've touched on. "Object Oriented" programming (OOP) takes functions one step further and groups them together into so-called 'objects'. To run a function you must ask the object which contains it. Objects have a type (a "class"), which can be a subclass of another class. For example "dog" might be a subclass of "animal", so that every function which "animal" can do ("breathe", "eat", etc.) a "dog" can do implicitly. Every object, for instance "Lassie", is an 'instance' of its class, for instance "Dog". Objects can also implement different roles (called "interfaces"), like "Lassie" can implement "Entertainer" and "Pet" (since not all "dogs" are "entertainers" or "pets"). By dividing a problem up into the different classes and objects it involves, the program becomes as simple as describing how they interact (eg. "lassie.fetch(big_stick)"). Learning the Object Oriented paradigm will make your life a LOT easier when you start to make large programs, but it is not very easy to do on your own.

5) EXPERIMENT!

If you have an idea or a question, try it out and see what happens! You can ONLY learn through experimenting and playing, there just isn't another way to do it.

6) Learn more languages

Scheme isn't the only kid on the block. There are many other languages you may like to learn. Good choices would be Smalltalk, Ruby, Python, Javascript, Vala and, if you're feeling masochistic, C, C++ and Java. The more languages you learn, the more approaches you learn to tackle problems with and the more you learn about why certain languages do things in certain ways (and therefore how to use them more efficiently). What may be hard in one language may be easy in another, so having a substantial set of knowledge is always a good thing.

7) Contribute to Free Software!

It might seem a daunting prospect to make your own Web browser, file manager or music player, but there are already many such projects out there with the code fully available for you to download and play with. Every Free Software project welcomes new developers, even if you're just fixing a bug which really annoys you. If you contribute to a widely-used piece of software like Firefox or Gedit then your changes, if approved by the maintainers, will be used by millions of people worldwide. If your changes aren't approved, but you would still like to make them available, you can keep them in a distributed version control system like Git or Bzr, letting you cherry-pick the upstream changes you want, and keep any changes you make yourself, and also let other people get your modified version.

Most importantly, don't be scared by all of the complicated stuff you see computers doing every day. Computers don't get harder and harder to program over time, they get easier and easier, since all of that fancy stuff has *already been written*! You might not understand how an image gets from the hard drive to the screen, but you don't need to know or care (I don't!), other people have already written libraries to do that. You just have to say something like "display(my_photo.jpeg)" and voila!

With computers becoming more and more integral to our way of life, I encourage everyone to at least attempt following some of those steps, if only to realise that computers aren't unknowable, magic things; they're understandable, controllable and most importantly they are changable. Posessing an attitude (like my Mum) of "I don't understand these things" is retarded. I didn't know how to piss standing up until I tried it.

Thursday, 18 September 2008

...And Laws Were Most Numerous When The Commonwealth Was Most Corrupt

Personally, I would say that power corrupts more than TV, miniskirts or saggy jeans.

I do, however, completely agree with Nsaba Buturo's statement "What's wrong with a miniskirt? You can cause an accident because some of our people are weak mentally,". He could've given an example if there was a mirror nearby.

Wednesday, 17 September 2008

The Effects of Promoting Propaganda

Richard Stallman and the Free Software Foundation are often the butt of jokes due to their campaign of explaining "GNU/Linux". The problem I have with their tactics is that they bring the issue up where it isn't particularly appropriate, for example in the recent Steven Fry video. There was no need to blabber on about such unnecessary bike shed issues in a video that is solely written by the FSF (ie. there's no need to correct anything, since they control its contents). I do, however, agree with the basic reasoning they have, that the phrases and words we use can shape our minds, and especially the minds of those we are talking to (if they are new to a subject).

In their case it is the use of the name "Linux" to describe an entire computer system, where Linux is only the kernel, and the use of the phrase Open Source instead of the phrase Free Software since the former promotes an efficient way of programming whereas the latter promotes the freedom of computer users. An example more common to people who aren't me would be the gross misuse of the word "pirate", used instead of "copyright infringer" in a context which implies that infringing copyright is akin to looting, murdering and raping on the high seas, and thus biases the minds of those who hear the phrase (murder, theft and rape are obviously wrong and criminal, and very serious matters indeed, whereas not only is copyright infringement an intangible and often debatable issue, it is not clear whether infringement is even wrong, despite its status as criminal).

One term a lot of people who know me will know my chargrin at is "secured" and "unsecured"/"insecure" when describing an 802.11 wireless computer network. The phrase is used to describe WEP and WPA encryption, but completely biases the minds of people who use and hear it. Deciding whether to allow or disallow devices to connect to a network is a matter that can be debated by rational people, whereas framing such a debate into choosing a "secure" network or an "insecure" network can easily end all discussion in favour of the option labelled "secure", ie. restricted access.

Such framing is plain wrong: First, a Virtual Private Network (VPN) can be used over any network, even an "unsecured" one, to establish secure communication between machines. For instance, a wireless access point can be used without encryption, thus allowing anyone who wants to to connect, whilst certain computers (for instance the tennants in a house) can use a VPN on top of this network to allow each other to access shared files and networked printers and nobody else on the access point can access things. A network could even be set up to allow the VPN as much bandwidth as it wants, giving what's left over to anyone else (so the tennants wouldn't even realise other people are online, for example, whilst people in the area can get free access to the knowledge and commication abilities that the Internet provides, although at a reduced priority).

Second, WEP encryption was broken years ago, thus any network which is "secured" by WEP actually has no security at all. Any device can use a Free Software tool like Aircrack to monitor the network automatically until it gets enough data to present the user with the encryption key being used.

The reason I'm going on about this is a very serious one: It might soon become illegal in India to allow people to connect to an 802.11 wireless computer network. Now, of course those behind this aren't using those words, they are saying "unsecured" networks will be illegal.

Oh yeah, it's also because those who use "unsecured" wireless networks are apparently terrorists. Since the T-word is in there I wouldn't be suprised if this does become law and sets an unnerving precedence.

BTW, I just noticed this (both by Uli Kusch though):





You may need to install SWFDec to watch those videos.

Friday, 12 September 2008

I am teh awesome

Seems I was overly cautious in my last post, since node deletion now works :D

Next I think I'll work on affiliations, which might take some thinking to do sensibly...

The Power To Change

As I've blogged about before, I'm currently working on a general purpose Python library which implements an entity in the XMPP/Jabber Publish/Subscribe specification XEP-0060. The library itself can currently send equivalent stanzas to all of the examples given in the specification. It does not yet handle the replies from everything, doesn't handle incoming messages (ie. updates) and its error handling consists of printing the error stanza to the terminal. However, at around 1,500 lines it's already a bit too much to grasp with only a specification to go on. For this reason, once I had implemented every needed method I decided to take a pragmatic approach to the reply and message handling.

This originally started as a PubSub reader application, similar in form and function to current news reader applications like Akregator and Liferea but using PubSub nodes instead of RSS or ATOM files. This proceeded quite nicely until I hit a bit of a wall: There wasn't anything to read!

To fix this I started to write a PubSub writer application, similar in form and function to current offline blogging applications like KBlogger and Gnome Blog but once again using PubSub nodes instead of RSS or ATOM files, or Web service APIs. This was a bit premature of me, however, since there was nowhere to write things TO, and then I came across x60br and thought it would be the perfect kind of tool to make next, thus I started a simple PubSub browser.

The concept of the PubSub browser I am making is simple, and I have blogged about it before. You're presented with a window containing minimal controls. Into a text box at the top the address of a PubSub-capable server is put, then the "Get" button next to it is pressed. The browser then adds this server to a tree view and uses the PubSub library to send a message to the given server, asking for a list of the nodes it contains. When a reply is received any nodes are added to the tree in a level below the server. Each of these nodes is then queried for any nodes that they may contain, and when the replies come in they are added to a level below their respective parents. This isn't quite a perfect way of doing things since it doesn't take into account the loops which the PubSub system allows, but is good enough for this tool's purposes.

With a tree of nodes available the next step to take was implementing some ability to alter the structure, ie. add, remove, change parents, etc. I've now achieved this, as can be seen in the following screenshot of my current ejabberd server.

Until this point the choice of server hadn't mattered to me (it's all standardised), and since ejabberd is a) packaged in Debian and b) all the rage these days, I figured it was worth using. There are currently VERY few servers which can actually understand PubSub, the most popular being ejabberd and OpenFire (the former being used by jabber.org and the latter being used by the University of Sheffield), but since it's such a new technology there are issues arising with its server-side implementation (which my library is not designed to address). Essentially ejabberd is currently a bit broken, the most important deviations from the standard being that it allows items to be published to a collection (analogous to writing text to a folder, rather than to a file inside the folder), it allows leaf nodes to contain children (analogous to a file containing other files and folders (OK this is possible with archives, but that's not the point :P )) and it doesn't allow subscribing to a collection. Aside from those, a rather obvious restriction (although still standards-compliant, as far as I can tell) is that a rigid filesystem-like structure must be adhered to, with node names following the structure /home/servername/username/... This means that pressing the Add button when the server, /home, /pubsub or /pubsub/nodes is selected will result in a failure, and with /home/localhost selected only one name is allowed, /home/localhost/test1 (since the browser is logged in as the user test1). However, it does work as can be seen (I've created the node /home/localhost/test1), and in the process I've managed to delete quite a lot of code from the library (note for non-programmers: The more code you can throw away, the easier your life becomes :P )

Next on the agenda is deleting nodes. If I can't get that done today, however, then I probably won't get it done until Wednesday. I'm going to see Jooooooooooooooooooooooooooooooooooooooooooooooo...ingtons.

PS: If you want to get the code there's a little difficulty at the moment, since I can't seem to sign anything :( I've set up a git repository here, but at the time of writing it's empty due to the aformentioned difficulty.

Thursday, 11 September 2008

Model-View-Controller

First there was Last.fm's site redesign, now there's Facebook's. Both have been given a hard time, however I don't personally give a flying fuck about their website design. The *ONLY* useful thing from last.fm is the artist information. Fuck the personal statistics. Fuck the Shoutboxes. Fuck the Friends. The *ONLY* reason accounts exist is to prevent spamming which would skew the artist information database. So, do you want to see how last.fm REALLY looks? Well there are a few styles: text and XML. There is also an API (Application Programmer's Interface) available, but it requires an account to use so it can go and die in a bin.

If you are particularly clueless about how computers work then you might be thinking to yourself "those look like shit", and you would be right. However, structure is FAR more important than presentation, and those documents are well structured. The reason thinking along those lines is clueless is because such people have never seen a computer program, they've never pressed the View Source button in their Web browser and they've never taken apart any gadgets in their life. Those of you who have done at least one of those things will know that beauty is only skin deep. A website might look pretty, but the HTML the pages are made of look as ugly as sin. The HTML will have a decent structure though, which means a Web browser can be stuck in between the user and the HTML to make it look pretty.

What I am saying is that last.fm allow access to their database in a structured way, which allows applications (which are completely stupid and need to be told exactly what to do, hence the need for structure) to display the data in whatever the hell way the user wants. You don't need to use their website, since code for putting data into and getting data from last.fm exists inside every decent music player (Amarok, Banshee, Listen, etc.). Your choice of application is completely up to you, and you can keep using that application for as long as last.fm's web services maintain their current structure. If the structure or protocol or something changes, then that's not too bad for anyone using a well written piece of Free Software. If you're using a proprietary program to display it then you're knackered and I hope you've learned a valuable lesson.

Now let's look at the recent Facebook change. Can I access Facebook's database via a well defined and structured interface? Can I bollocks. That means I'm stuck with whatever the almighty Facebook deities bestow upon me, and I'd better pray that I like it because it's all they allow me. Thank fuck I don't use it.

For those of you who haven't realised it by now I am talking about the Semantic Web and the Model View Controller architecture. In the World Wide Web the stuff that gets passed around is HTML. That HTML can contain any data, is layed out in a certain way and the structure is very freeform, as long as it meets a few rules defined in the HTML and XHTML standards. The Semantic Web is different. In the Semantic Web the data is structured in a very specific way, in RDF triples to be precise. There is NO layout in the Semantic Web, since it is not about anything visual like pages, it is about *concepts* and *meaning*. Applications you use which can access the Semantic Web (there are as many as people can create, rather than the Web's singleton the Web Browser) can do whatever the hell they like with the structured information they receive. There's no arguments about the layout and look of a semantic Facebook because there's NO SUCH THING as the "layout" or "look" of anything on the Semantic Web. The layout and look are ENTIRELY up to the user and which program they decide to view it with. Some users may even view it with applications they access via the World Wide Web.

PS: I would just like to point out the first sentence of Wikipedia's World Wide Web article:

"The World Wide Web (commonly shortened to the Web) is a system of interlinked hypertext documents accessed via the Internet." Thank you.

Sunday, 7 September 2008

Services, integration, communication and standards

These days it is possible to do pretty much anything over the Internet. There's eBay, online banking, PayPal, Flickr, OpenStreetMap, OpenDesktop, Email, chat, forums, Wikis, Facebook, scrobbling, blogs, etc. The big problem, in my mind, is that of technological walled gardens.

A walled garden is an area which cannot be accessed without permission, usually by registering some form of account. Facebook is a classic example of a walled garden. Walls are useful, since having everyone's bank accounts available without restriction would be a problem to say the least. A technological walled garden would be an enclosed, restricted area which can only be accessed via certain technological means. Technological walled gardens are often simpler to implement than open systems, but often the reason the garden operator does this is because they see this as a way to run a dot-com or Web-2.0 business.

Let's take an example, Yahoo! Mail, Windows Live Mail and Gmail, which are all walled gardens in the classical sense, an account is needed and the login details must be provided in order to access anything. The first two, however, are also technological walled gardens: whilst mechanisms to send, retrieve, check and manage email have been around for decades, from "get my mail" (POP3) and "send my mail" (SMTP) to more sophisticated "synchronise this lot of email with that lot" (IMAP) and are well defined, standardised, understood and implemented, in order to access Yahoo! Mail or Windows Live Mail you still need to log in via their website because they don't use any of these standards. Gmail supports them, which is how I can use Evolution and Kmail to manage my Gmail account. Yahoo and Microsoft specifically disable them (I know Yahoo used to allow POP3 and SMTP access, when they stopped I moved away from them) with the reasoning that Evolution and Kmail don't display adverts, whereas their websites do. Here interoperability and standardisation desired by customers (if it wasn't used then there's no point disabling it, since nobody would be unexposed to adverts and the POP/SMTP/IMAP server load would be zero) is sacrificed in order to force adverts onto users who don't want them. This of course doesn't even touch upon the flexibility of using an email client (screen readers and other accessibility for the disabled, offline access, complete choice of interface (including Web), etc.).

That is the major reason why I refuse to join Facebook, MySpace, etc. I cannot buy or download Facebook's software and run it on my own machine,and even if I managed to write my own there would be no way to make it talk to Facebook's own servers. Since the entire point of Facebook is the stuff in their database, this would make my software useless. Hence Facebook have created a technological walled garden: If I joined Facebook then I would be sending any data I entered into a blackhole as far as accessing it on my terms is concerned.

Last.fm is better, since although their server/database software is a trade secret (as far as I can tell), the Audio Scrobbler system they use to gather data is completely documented and has many implementations, many of which are Free Software (including the official player). The contents of their database is also available, and doesn't even require an account (I have tried to think of ways to submit similar artists without an account, such as submitting two tracks at a time and building the data from that, but I suppose spam would be too much of a problem). Only the artist recommendations/similarity, tags and thingsare available, but that's the entire reason I use it, fuck the site with all of its Flash, poor social networking, confusing messaging system and stuff, that's all fluff around the periphery of the useful information. Essentially last.fm is like Gmail: I can't get the code which runs it, but I can get my data in and out in standardised ways which can be implemented with Free Software. I could make my own server which synchronises with with their database via the available protocols, and thus get out everything that I put in.

Now, the issue of synchronisation is interesting. How do you keep two things synchronised? There are a few different approaches and each has its place:

Known, unchanging, unmoving data

Here HTML can be used, ie. a web page. This is fine for people, and for applications needing that data it can simply be copied into the application once. An example would be an "about" page.

Unknown, unchanging, unmoving data

Here HTML can still be used, but since the data is not know beforehand it can be hard for an application to get anything useful from it. RDFa can be used inside the HTML to label each piece of information, thus an application only needs to be told what to find and it will look through the labels until it does, regardless of page structure, layout, etc. An example would be a scientific paper.

Changing data which is accessed once at a time

Here RSS or ATOM can be used. This allows changes to be specified in the file. ATOM is a standard, but RSS is a dialect of RDF which means labelled data is possible. An example would be a changelog.

Changing data which is accessed on every change

Here XMPP PubSub can be used. This means that there is no checking for updates since the source will push any changes out to subscribers when they are made. This doesn't use a file, it uses a stream. This is what my library is designed to accomplish. An example would be a blog.

Two-way communication and instruction execution

Here systems such as JOLIE can be used, overlaying protocols like SOAP. This can be used for dynamically generated data like database queries and searches, as well as for sending instructions such as "Approve Payment". An example would be a shop.

Notice that the first technology, HTML, is the only one which needs a Web browser to access. RDFa, ATOM and RSS are all structured in a way that allows applications to handle them directly, no human is needed and thus many more possibilities are available. XMPP can also be structured, since RDF, ATOM and RSS can all be sent over XMPP, allowing machines to handle the data, but doing so in an as-needed basis which makes things more scalable. JOLIE covers a range of protocols which are all inerently machine-focused, they are executing instructions. This might be a "buy this laptop" instruction when a button is pressed, or a "search for this" instruction when a query is entered.

These technologies allow data and communication to break free of implementation and visualisation. The next Facebook doesn't have to be a centralised Web site, it can be a distributed system with many servers run by different people interacting with each other to allow a scalable, unowned network, like the Web but made of pure information without the overhead of layout, styles, widgets or interfaces. This is called the Semantic Web. All of the visualisation, interface and style can be swappable and implemented where it is wanted, for instance as desktop applications on user's machines, or as Web sites running on various servers. There is no reason why, in an interconnected world, I should have to visit Facebook.com in order to interact with Facebook.

Except, of course, that Facebook wants to force unwanted adverts onto their customers.

NOTE TO SELF

When trying to communicate from a machine to itself, for example trying to run the ejabberd server, it is a good idea to have a loopback network interface. Spent over a week trying to hunt that issue down, now I can finally get back to working on my library :D

Friday, 5 September 2008

Physics + Programming = Pretty

Whilst looking for a better-than-mine Python implementation of attractive-and-repulsive-based-on-the-inverse-square-law particle simulation I came across this, which is basically a really simple model of gravity. You tell it how many balls you want it to start with, which are then placed at random points on the screen with zero momentum and a mass of 4. Then the simulation starts, running one time step of the simulation and redrawing the screen, then repeating this indefinitely. Particles which collide coalesce into a single particle with the combined mass and momentum of the original. This results in one of three things happening, either all of the masses coalesce into a big, stationary ball (since the net momentum is zero, due to the law of conservation of momentum), two or more masses form a somewhat stable orbit around each other, or one or masses hit the edge of the window during the simulation and 'reflect' (the vector of their velocity perpendicular to the edge is reversed) which breaks the physical simulation by violating conservation of momentum. After this third scenario all hell can break loose, usually resulting in everything coalescing into a huge ball, moving with momentum twice the magnitude of that which was flipped by hitting the wall, and opposite in direction.

Anyway, I looked through the short source file and had a bit of a play. Eventually I gave particles a colour which changes subtley every time they are drawn. I also disabled the old-frame-removing screen fill, allowing each frame of the animation to build upon the previous and create a trail. A little simple colour-cycle logic and I the following patterns appeared to my delight :D




HERE'S THE SOURCE (not properly licensed due to original author. Let's say you can treat is as Public Domain to save complications of licenses)

Coming down to Earth with Physics

I read a blog post today where the writer imagined future computers being capable of unlimited calculations, storage, etc.

To quote the author: "Electricity and other force waves can be “on” or “off”, but they also have lots of other properties, many of which are sufficient to represent an infinity (or near-infinity). Frequency of any wave represents an infinity, for example–you can have a 1 Hz wave, a 1.1 Hz wave, a 1.15 Hz wave, a 1.151 Hz wave, etc.".

Disregarding the nonsense term "near-infinity", here an assumption has been made that waves have a continuous spectrum of frequencies, so that another decimal can always be added and allow ten times the number of frequencies to be used. Is this assumption valid? Physics says no. The frequency of a wave is equal to the energy of the wave divided by Planck's constant (f = E/h). Since Planck's constant is constant, for a wave to have a continuous spread of frequencies it must have a continuous spread of energies. Quantum mechanics shows us that energy occurs only in certain quantities ("quantity" making it "quantum"), for a very simple example see the quantum harmonic oscillator.

The author also states "the fact that waves take up essentially no space (only the medium that they vibrate takes up space)." To me this represents a complete misunderstanding of what a wave actually is. Wikipedia defines a wave as:

"A wave is a disturbance that propagates through space and time, usually with transference of energy. While a mechanical wave exists in a medium (which on deformation is capable of producing elastic restoring forces), waves of electromagnetic radiation (and probably gravitational radiation) can travel through vacuum, that is, without a medium. Waves travel and transfer energy from one point to another, often with little or no permanent displacement of the particles of the medium (that is, with little or no associated mass transport); instead there are oscillations around almost fixed locations."

Notice that two contradictions to the blog author appear here. Firstly the definition includes, straight away, the word space. Without space there are no waves. Secondly waves do not necessarily need a medium to exist in. A wave is not an entity of itself, it is an abstract concept which describes the oscillation of something.

Add to this the fact that analogue systems (ie. continuous valued measurables) suffer from noise in a non-recoverable way (unlike digital systems), which makes your measuring device limited to its resolving power, which once again is limited thanks to the effect of apertures (the smaller the worse, by the way, making smaller spaces less distinguishable), and this whole concept, whilst initially sciencey-sounding, is unfortunately a no-starter.

LOLFOX

Made this when I saw this. I couldn't find any link back to the original image on that derivitive so I googled for "fox mouse", and this appeared as the fifth result, proving once again that rule 34 is alive and well :D

Efficient vs Approachable (choose your phrase carefully)

I am very interested in human computer interaction (HCI), and I like to follow the trends and experiment with new innovations, plus I try wherever possible to commend developers who make different and interesting things, regardless of whether they are any good or not, since experimentation is key to finding better ways of doing things than we have now. Afterall, the same processes which turned slime into us are responsible for birth defects and fatal genetic diseases, you can't have the good without the bad.

I find a big problem with the phrase "easy to use", since it covers many varied situations. "Easy to use" and "hard to use" can both describe the same thing at the same time, since the words used are so vague; "use" covers a lot of ground, as do "easy" and "hard". Therefore I try where possible to use more specific words. The most common ones I use are efficient/inefficient and approachable/confusing. Why are those more descriptive? Here's an example: Compare the Vi text editor to the Leafpad text editor.

Vi is easy to use since its keyboard commands mean you never have to take your hands off the keyboard. All formatting, editing and control functions can be accessed the same way, via a few button presses. This could also be called efficient.

Vi is hard to use since it doesn't have any buttons and nothing is labelled. The whole interface must be learned and you have to look in the manual just to find out how to quit. This could be described as confusing.

Leafpad is easy to use since buttons are labelled and organised in a menu. Different tasks are given different places, structuring the interface. This could also be called approachable.

Leafpad is hard to use since it requires a whole graphical environment just to start. To make selections and issue commands you need to constantly swap your hands between the keyboard and the mouse. Some things take longer to do than others, because they're buried in menus. This could also be called inefficient.

As you can see, both can be described as easy and hard at the same time, even though those words are mutually exclusive. That means the words easy and hard are wrong to use. Whilst we can't really draw much meaning from saying "Vi is easy but hard", we can say "Vi is efficient but confusing". Likewise we can say "Leafpad is approachable but inefficient". Both have their advantages and their disadvantages. To someone who spends a lot of time in a text editor, for example a programmer or a journalist, it would be sensible to invest the time to learn Vi. However, for a general audience Leafpad would probably be more appropriate, thus the default text editor on a mass market operating system should be Leafpad rather than Vi. The same argument can be used for Blender, since its own peculiarities make it very efficient for someone who knows what they're doing, but make it confusing and unwieldy for someone who doesn't want or need to spend the time learning the interface. The only difference here is that I don't know anything as capable as Blender to contrast it with.

Regardless of what experienced users and developers may prefer, the approachable option should always be the default. Saying "I prefer XYZ" doesn't particularly matter when you know how to use a package manager. The defaults should be chosen to suit those who don't know how to use a package manager, or even what one is, since those are the users who will be using the defaults no matter what (because they know of no alternatives). Please make their lives easier.

The end.

Thursday, 4 September 2008

Last.fm Is Becoming More Accurate

XMMS (the X MultiMedia System) was the first music player I used when I switched to RedHat. Its support for ProTracker via ModPlug was awesome, whilst MP3, Ogg Vorbis and even WMA worked (although the latter required a hard-to-find plugin RPM), but it now seems to be losing mindshare. The software was essentially declared finished quite a few years ago, and now bug fixes seem to be dwindling and it is even dropping out of some repositories. However, as we all know Free Software gives everyone the right to fork, and XMMS is well and truly forked!

XMMS begat Beep Media Player, and BMP begat BMPx and Audacious, both of which are doing fine. After just installing Audacious, and enabled its Last.fm plugin, I can now enhance the accuracy of my profile by submitting the Tracker and Video Game Music files I listen to. These were music to me from when I was 4 all of the way up to the Napster era (which was followed by the Kazaa era, the Kazaa-Lite era, the eMule era, the aMule era (when I switched to RedHat) and finally landing in the MLDonkey era, which is actually a superset of the previous :D ), thus I my Last.fm profile is incomplete without them.

UPDATE: Last.fm is being an Epic Fail and requiring some kind of structured metadata, no modules don't seem to be working :(

Tuesday, 2 September 2008

On Responsiveness

Modern computer systems are multitasking, multiprocessor, multiprocessing, multithreaded powerhouses. Why does an application's user interface become unresponsive when it is doing a task? This seems like a bad design decision on the part of the toolit makers. Programming in a toolkit like QT or GTK+ is already asynchronous, why doesn't the UI keep itself up to date?

For the record, AmigaOS gave its user interface a higher priority than the applications, so that if the user moved the mouse then no matter what the Amiga was doing it would stop and update the cursor position then carry on, the same for buttons being hovered over and depressed, etc. This makes a 20 year old machine *feel* more responsive than current state-of-the-art. My 2 year old laptop with a 2 month old desktop system makes me ANGRY at the scrollbars for not moving. That's just not on.

What I Find Cool

The Winter Gardens has a free Wifi access point with full Internet access. There are also two T-Mobile ones, but who gives a shit about their pay-per-use crapware anyway?



(Too bad they recommend Skype for free calls :( )

Sunday, 31 August 2008

OMFG I want to be there

Free Software, Physics, programming, One Laptop Per Child and Europe. Want.

Wednesday, 27 August 2008

C00l

Suspend actually seems like a nice feature. Never tried it before, but I'm guessing it would be useful when I need to carry my laptop somewhere in my backpack for a while, but not long enough to warrant turning it off. Don't know it if worked straight away or if it was a quirk I used to try and get hibernation to work. (Which still doesn't).

Also, why do so many people get food from John's Van? It looks and smells like cat sick. I don't see the appeal.

Note to self

If ALSA screws up again, make sure to check for hidden configuration files in your Home folder as well as the system defaults. Comment the entire lot out, since the autodetection is smarter than whoever wrote those files.

Finally, Ayreon are back :D

Resits Over

Now it's in the hands of the Gods (by which I mean, of course, the Scientists). There's nothing I can do to change the outcome now, but I at least hope that nobody seriously pisses them off before they get marking.

COM166 wasn't too bad. It was actually the same paper as the first time from what I could tell, but the papers had to be left behind. PHY203 was allowed to be taken away after the exam, thus they had a different paper for the resit. Weirdly I seemed better at the statistical mechanics (Boltzmann distributions, partition functions and the like, but sadly not so much with the statistical weighting of macrostates and its relation to entropy :( Yes, I know the formula, but just couldn't articulate its relation to microstate counting very well) than some of the thermodynamics, even though I left it very late with the statistical revision. Meh, probably it was fresher in my mind.

Anyway, yes. They're gone now and I'm confident I've passed, but obviously we live in a Universe who's determinism is a matter of debate. I may have done slightly better had I not been listening to the BRAT game music playing over and over in my head on the way into the exam, and a subconscious yet rather annoying repetition of "WOOOOOHOHOHOOOOOAAA!" about half way through. Yes, my head is weird. m8.

Now I can carry on exactly what I was doing before, but this time not feel so guilty about wasting time :P </joking>

Two Wrongs Do Not Make A Right

Recently there was an Open Source iPhone application made called OpenClip which allows copy and paste functionality to work between applications. Well, seems Apple didn't care for it and in their latest firmware update they've stopped it working. To me that just means a bit of I-told-you-so gloating, since it's exactly what to expect when you take on a computer that's largely controlled, and for all intents and purposes owned, by the manufacturer.

In somewhat related news, perhaps this is Karma (no not the OpenSuse one), the iPhone adverts which Apple have been spreading liberally all over the place have been labelled "misleading" by the Advertising Standards Authority. Now, although it causes yet another I-told-you-so moment and I'm obviously teh awesome at modestness, I'm going to tell you what it's about. The adverts are apparently misleading because they claim "all the parts of the internet are on the iPhone", but since Flash and Java aren't installed then that claim is wrong and (to quote the BBC) "Some pages on the iPhone won't look like they do at home".

Now, I'm sure a few people will remember my furious claims that the Internet and the Web are not the same thing. Well, this confusion is right at the heart of the issue. I'll say right away that I think this decision is wrong and that the iPhone adverts are not misleading in the way that has been claimed. I'll detail why it is wrong below:

Teh Internets

The Internet is a super-global computer network. A computer network is defined (as I can attest from both my formal and informal studies in this area) as a way for machines to communicate, but which specifically describes a general network (so the 'phone network isn't a computer network since it passes analogue audio signals around, whilst most computer networks pass packets of binary data which are, in a quantum mechanical world, general as they can mean anything). An internetwork (an internet) is a network of networks, where devices called routers translate between the different physical technologies and virtual addressing systems of each network (for instance I have a router in my house which translates between IEEE 802.11g wireless networking, Ethernet, USB networking and my ADSL line). "The Internet" is a slang name for the network of internetworks which reaches around the globe, into the Earth and out into space which is more commonly referred to by its correct title "teh Internets". If you want a diagram showing the "net" then this'll have to do:
As you can see, each vertex (joint) is a computer, and each edge is a network connection of some kind. Apple have claimed that on their iPhone (connected to the 3G network operator on the left at about 8 o'clock) has all parts of the Internet on the 'phone. Now, since the Internet is by definition more than just one 'phone this obviously referrs to the fact that the 'phone can access the entire Internet (insomuch as any device can, taking into account people using encryption and various server outages around the world). The Internet is real, physical thing that you can see if you look at the wires coming out of your house. The iPhone can access the entire Internet just as much as a desktop computer.

Now, the actual data that gets passed around on the Internet can be literally anything, since it is by definition a general network. However, throwing random bits at computers isn't particularly useful if the other end doesn't know what you're trying to say. This is where protocols come in. There are LOADS of protocols, and they stack up on top of each other at various layers of the architecture. Examples are IP (the Internet Protocol), FTP (file transfer protocol), XMPP (eXtensible Messaging and Presence Protocol), SMTP (simple mail transport protocol), and of course HTTP (hypertext transfer protocol).

The Web

HTTP is all about transfering "hypertext". This is like plaintext (which is actually just a bunch of binary, anything can be seen as a type of plaintext) except that it uses SGML markup to specify links (called "hrefs"). These links allow readers of the hypertext document to be transported to another hypertext document living at the address given in the href. I'll draw you another diagram below:
This is a little harder to understand since there are two kinds of node, the pages (in green) and the links, technically called anchors (in red). Plus the edges are directed (ie. there are arrows). Just to clarify, there are no arrows pointing TO links, all arrows end in a green page but I'm not too good at drawing with a mouse :P

What's happening is that you're at a certain page in your browser (in green) when you click one of the links (in red). Your browser shoots off down the arrow coming from that link until it gets to the green page at the end of it. If there are links in that page you can click those, but if not then you either need to go Back along the last edge your browser went down or enter a location manually. This is the Web, and you can't see it because it's not real. It's just a visualisation of a data structure. The green web pages mostly live on computers , which are nodes in the first picture, but the Internet and the Web are completely different things.

Web pages, as I've said, are hypertext, written in the HyperText Markup Language (HTML), which is just a special structuring of text. HTML is designed to allow arbitrary non-HTML stuff to be stuck inside, and the standards say, and I quote:

13.3.1 Rules for rendering objects

A user agent must interpret an OBJECT element according to the following precedence rules:

  1. The user agent must first try to render the object. It should not render the element's contents, but it must examine them in case the element contains any direct children that are PARAM elements (see object initialization) or MAP elements (see client-side image maps).
  2. If the user agent is not able to render the object for whatever reason (configured not to, lack of resources, wrong architecture, etc.), it must try to render its contents.

Authors should not include content in OBJECT elements that appear in the HEAD element.

Flash and Java applets are examples of OBJECT elements. The standards say that the user agent (browser) should try to render OBJECT elements which aren't inside the HEAD element. If they can't then they should try to render the contents (for example the "alt" text which is rendered if an image doesn't load). The iPhone's WebKit browser is completely following the rules here.

So, in essence what I am saying is this:

1) The "Internet" is the physical network which carries data. The iPhone has a complete connection to the Internet, as they claim.

2) The Web, which seems to be the point being argued, is NOT the Internet. Apple have not claimed that everything on the Web will work on the iPhone, they've said that you can get everything that is on the Internet.

3) Flash and Java are not HTML and thus not even IN the Web. They are external in nature, even if they appear in the middle of a HTML document. For an analogy, open a text editor window and drag it over this web browser window. Is the text editor now part of the Web? Is it bollocks. Flash and Java can communicate over HTTP if directed to do so but they are not part of the Web. They are interpreted programs which can be accessed over the Internet.

4) Flash and Java have been incredibly proprietary until very very recently. When the iPhone was in development it was completely reasonable to not include these proprietary technologies, especially if they weren't available for Darwin on an ARM (which was an internal Apple build environment until the iPhone came out anyway). Now that Free Software Flash players and Java environments are getting mature it may be a different story, since the rules are different with Free Software (Apple would still be in control, rather than Sun or Adobe).

5) For fuck's sake! The *BEST* part about the Web is that it degrades gracefully. If a browser doesn't support some feature which it encounters then it just ignores the request and carries on. This is exactly what the iPhone is doing. The real irony is that WebKit, the browser engine used on the iPhone, is one of the most standards-compliant, correctly done browsers there is! The majority of desktop machines are stuck on Internet Explorer, so the comparison to "at home" (ie. on a desktop or laptop, most probably running IE) is actually scary.

OK rant over. For now.

Tuesday, 26 August 2008

An Option I Would Like To See In Every Application

It runs like ass on my graphics card+driver combination (and even pops up a message saying that my setup is unsupported) but at least it's running! That's OpenMetaverse Viewer, previously known as SecondLife Viewer. Linden Labs are trying to make SecondLife completely Free Software, their client has been for a while and their servers are completely Debian-based, but due to a) proprietary third-party code they're not allowed to liberate and must replace, and b) their desire for every server to be able to communicate with each other (and thus not fracture the users into camps) it is taking a while.

Some particularly zealot-like Free Software people have made their own server called OpenSim, so to try and get rid of trademarks/company names/etc. the Free Software SecondLife viewer is now OpenMetaverse Viewer and can connect to SecondLife and OpenSim servers.

Since Ubuntu was being a bit of a pain I've gone back to Debian, unstable to be precise, but if you want to install omvviewer (found here) then you'll need some packages from testing which can be found here. Specifically you'll need "xulrunner", "libxul-common", "libxul0d", "libnss3-0d" and "libmozjs0d" which are all to do with Mozilla's XULRunner (the thing Firefox is made in) since that can apparently be embedded in the virtual worlds.

Sunday, 24 August 2008

On Computer Animation

Animation is an optical illusion brought about by rapidly changing images and the brain's natural appreciation of Heisenberg's Uncertainty Principle.

OMFG!

Teh awesome!

Saturday, 23 August 2008

QEdje Is Working! :D

There are quite a few Free Software libraries out there for making GUIs (graphical user interfaces). The most famous are probably GTK+ (started by the GIMP and used by Gnome, XFCE and LXDE) and QT (started by Trolltech and used by KDE), however there are quite a few more such as GNUStep, Motif (used by CDE), FLTK (used by EDE) and EFL (used by Enlightenment).

EFL, the Enlightenment Foundation Libraries, are particularly interesting. They are incredibly lightweight, running quite happily on a mobile phone, yet allow all sorts of animation (as in, proper 2D animation rather than just moving/spinning/stretching things) and are completely themable. The way this works (from what I can find out) is that every EFL program uses the Evas canvas ("canvas" is the name given to a widget which allows arbitrary drawing on top, rather than imposing some kind of structure), then Etk and EWL draw on top (the canvas is created implicitly by Etk and EWL, even if you don't make one explicitly). This is opposite to most toolkits, like GTK+ for example, where the widgets are drawn in the window (which is usually divided up into a rigid structure) and canvases are implemented as widgets.

A nice feature of the EFL is called Edje. Edje allows an application to be written without worrying about the GUI, instead just requiring that an external file be called. These external files describe the interface, and are entirely declarative (that is, they say "I want a button" rather than "This is how to draw a button"), think of it like the HTML of a web page, with Edje being the web browser which draws it (actually, this would be a more appropriate description of XUL, but I can't get my head around XUL's seemingly overcomplicated use of CSS, JavaScript and XML :( ).

Edje files are compiled into compressed archives (using EET) which act like incredibly far-reaching themes. This means that a theme doesn't just contain pretty pictures to use as buttons, or the programming to draw the right lines at the right locations, it actually contains the entire user interface. To continue the web page analogy, if Gmail or Facebook used an analogous system then instead of merely being able to change the theming via CSS (which may have to be specifically forced from the browser preferences, because web-app people suck balls), you could actually use a completely different webpage to interact with the underlying application (no more "New look!" announcements, since anybody could use any look they wanted all of the time).

Now to address the title of this post ;) As I've described, Edje is a declarative system. An awesome feature of this is that Edje can be completely replaced without the user even noticing, since the point is to say "I want a button" and not care about how it gets done. Well, the developers of Canola looked at moving to QT, since it offers more features than EFL, is more widely developed, developed for and installed. However, they found that Edje was too awesome to leave behind, so they ported it to QT and called it QEdje! What's particularly nice about QEdje is that a) the canvas used instead of Evas, called QZion, is rather abstract in itself, so that different QT systems can be used to do the work (eg. direct drawing to the screen with the QPainter backend, more abstract layout with the QGraphicsView backend or 3D accelerated with the KGameCanvas backend, depending on the environment it is being used in) and b) the huge wealth of QT widgets can be used in the program (this is pretty powerful considering that as well as buttons, labels and tickboxes, QT also has a whole web browser, Open Document Format compatible rich text areas and an abstracted audio/video multimedia engine (which, so far, uses Gstreamer (Rhythmbox, Totem, etc.), Xine (GXine, xine-ui), VLC, Mplayer, QuickTime (QuickTime Player, etc.) on Mac and DirectShow (Windows Media Player, Media Player Classic, etc.) on Windows).

After a little wrangling I've got it to compile and the examples are working. This means I can have a play about, so I'll report on my findings :)

Friday, 22 August 2008

Reasons I Love Teh Internets

You're one badass fucking fractal

Thursday, 21 August 2008

You know you're a sad bastard when...


You get excited at this headline, then disappointed with the story.

Tuesday, 19 August 2008

Exactly

Saturday, 16 August 2008

A lengthy list of practical problems with shipping non-free software

This has been written since Ubuntu Brainstorm doesn't allow comments over a certain length, I want to cover this in as much depth as I can and I have a tendency to ramble :P

This is about non-free software (ie. proprietary, the cost doesn't matter) and the practical problems which would be encountered if it were shipped by default in a distribution. This does not go into the Free Software vs. proprietary software political arguments, it is all about the undeniable problems that come with non-free software.

The main software for discussion is codecs, the software which can encode and decode different types of file. Codecs are simply algorithms, they are nothing more. They contain no GUI, no player, nothing except the algorithm. An algorithm (like MP3) is just a set of steps to follow, as an example an algorithm to read a book would be something like:

Pick up book

Rotate so that front cover is facing you and the top is at the top

Lift the front cover at the edge furthest from the spine, being careful not to lift any pages

Turn the front cover 180 degrees until it is in the same plane as the pages yet on the opposite side of the spine

If there is any text on the inside of the page beneath the front cover:

- Find the end of the top line closest to the spine
- Read until the end closest to the edge of the book is reached
- If there is any more text on the page then move to the line below and repeat

Turn a single page from the right-hand-side to the left-hand-side of the spine in the same way as the front cover

If there is any text on the left page:

- Find the end of the top line closest to the edge of the book
- Read until the end closest to the spine is reached
- If there is any more text on the page then move to the line below and repeat

Repeat the steps for the previous page, then for this page, on every subsequent page until the end of the book is reached

Turn the back cover of the book onto the rest of the book

Put down the book

That's obviously a very crude algorithm as it doesn't take into account right-to-left languages, footnotes, etc. but it is a valid implementation of a book decoder. The copyright on the above text belongs to me, since I have written it, and will remain so for probably well over a hundred years. However, somebody can look at a book and make their own algorithm to read it without ever knowing that I've even written this blog post. My copyrights will not be infringed since their algorithm cannot be a derivitive of my work as they haven't even seen it.

This is how LAME (LAME Ain't an MP3 Encoder) was written. The developers didn't look at how anyone else had decoded the MP3 format, they didn't need to, and thus no copyright is infringed.

However, software patents allow me to patent the working of my algorithm, as long as it is novel and non-obvious to someone skilled in the field where it applies (the rigour for such things is often debatable though!).

The problem with patents on software is that it's very easy for someone to make software, but they could be treading all over other people's patents without ever know it!

Wednesday, 13 August 2008

Some ramblings about graphics, compilers and CPUs

This is probably very wrong, but whatever, it's *my* blog :P

I am following the work going on with Gallium3D as much as possible, since this seems like it should offer some pretty cool features when it becomes stable. It seems (from what I understand) to be an abstraction layer over the graphics card. Until now drivers are written for graphics cards which implement certain things like OpenGL, etc. and anything which is not supported relies on (much slower) software rendering (like Mesa for OpenGL), or if that's not available it just doesn't work.

Since the drivers are responsible for providing a lot of things as well as the basic hardware access this results in a lot of duplicate work, with slightly different ways of doing the same things for each driver (for instance every driver needs to implement a memory manager. The Intel driver's memory manager is called GEM and is Free Software and available for other drivers to use, but it's written in a very Intel-specific way and is therefore useless to other drivers). There is work going on to make a generic memory manager for the kernel and a generic way for graphics drivers to access the hardware (called the Direct Rendering Infrastructure, working towards version 2), but it still leaves rather a lot of common ground in each driver.

Gallium3D is a very generic abstraction layer that is designed to sit in between the drivers and the actual programs and libraries being used. Gallium3D is implemented to work in a very similar way to the actual silicon inside current graphics cards, thus writing a driver to sit between the card and Gallium3D is pretty straighforward because not much has to be translated from one representation to another (and DRI can still be used to do a lot of the background work). This makes Gallium3D act rather like a software graphics card, but doesn't cause too much slowdown since a) it is so similar to real hardware and b) it uses the Low Level Virtual Machine (LLVM) system to gain Just In Time compilation and optimisation for its code*. Since Gallium3D is software it can be run on any hardware with a suitable driver (which, as I said, is pretty painless to make), so libraries like OpenGL can be written to talk to Gallium3D and they will run on whatever hardware Gallium3D talks to (with software like Mesa running anything that the hardware can't handle). This means that writing a graphics library is a lot easier (since you only have to write it for Gallium3D's state tracker, eerything else comes free after that) and thus more than the basic OpenGL-and-that's-it can be used.

As an example, WINE can run Windows programs on Linux, BSD, MacOS, etc. (and even on Windows :P ). However, a lot of Windows programs and especially games use Microsoft's DirectX system for 3D graphics. Normally the WINE team writes replacements for Microsoft's libraries which perform the same job, thus allowing Windows programs to run, but writing their own DirectX implementation would be a huge job on its own. Making a different one for every graphics card driver out there would be even worse, and relying on the driver's developers to do it like they currently do for OpenGL would make the drivers EVEN MORE bloated and complicated than they already are.

Thus the WINE team are writing a DirectX implementation, but instead of working with the graphics card they are writing it to use OpenGL (since OpenGL is already in the current drivers). This is pretty inefficient, however, since DirectX operations need to be translated to OpenGL, then the OpenGL needs to be translated to however the graphics card works, then run, and since games are generally very resource intensive, not to mention that 3D is one of the hardest things for a computer to do anyway, it's far from ideal. With Gallium3D, however, the OpenGL translation can be skipped entirely and DirectX can be implemented straight to Gallium3D, which then gets sent very efficiently to the graphics card whilst still only needing to be written for one system. Likewise other libraries can be accelerated through a graphics card without fear of some users not having a driver capable of it. Application developers could even write a custom library for their application and count on it being accelerated regardless of the card it runs on (for example, a proprietary program cannot count on drivers containing code to handle their custom, proprietary library since it can't be included by its very nature. It can, however, count on Gallium3D being installed and thus can include one implementation which will be accelerated regardless of the driver).

* JIT compilation is pretty cool. There are historically 2 types of program: one is compiled (like C) which means the program is first turned from source code into machine code, and then this machine code runs when you tell it to. The second is interpreted (like Python), which means that instead of running directly on the computer, it is run instead inside another program called a "virtual machine" (which is usually compiled, although it can itself be interpreted, as long as at some point there is a compiled virtual machine talking to the real computer). Programs are more flexible than computer processors, which means that interpreted programming languages can have many nice features added and usually be written more simply than compiled languages. Compiled programs usually run faster than interpreted programs, since they are run directly, there is no virtual machine acting as a middle-man.

Just-In-Time compilation, however, is like a mixture between the two. The source code is NOT compiled into a runnable program, making it different to compiled languages. However, there is no need for a virtual machine, thus making it not an interpreted language. What happens is very siimlar to an interpreted language, but instead of the virtual machine looking at what to do next then telling the computer, in a JIT language the next bit of code gets compiled just before it needs to run and is then given to the real computer just like a compiled program.

At first glance it would seem like this might be faster than interpreting a language (again, no middle-man) but slightly slower than a compiled program (since with a compiled program all of the compiling is already done before you run the program). However, JIT-compiled languages can actually be FASTER than compiled programs!

This is because when a program is compiled then that is it, the finished thing, that code will be sent to the computer. Compiled programs must therefore be able to handle anything they are given, so a banking program must be able to handle everything from fractions of pennies up to billions and perhaps trillions of pounds. Such large numbers take up a lot of memory, and moving them around and performing calculations on them can take time. Compiled programs have to use a lot of memory for everything and accept these slow-downs just in case they are given a huge number. This means handling £1 as something like £0000000000001, just so it can deal with a trillion pounds if so commanded. A JIT-compiled program, however, knows how much it is dealing with, thus £1 can be stored as simply £1, whilst a thousand billion pounds can still be dealt with if it comes along by storing it as £1000000000000. This means JIT programs can use less memory for storing values, the calculations they do can be quicker as they don't need to deal with unneeded digits, and the program can speed up greatly due to less cache misses**.

Another advantage of JIT compilation is that unneeded calculations can be skipped. For example, a program may need to add deposits to an account and take away withdrawals. In a precompiled program this will always take at least two calculations, either add the deposits to the total then take the withdrawals from the total, or take the withdrawals away from the deposits and add the result onto the total. In a JIT-compiled program this can be made more efficient, since if the withdrawal amount is zero then the compiler can skip one of the calculations, and likewise if the deposits are zero. If both are zero then both calculations can be skipped, and if they are both the same then the calculations can also be skipped. For instance, compare the following precompiled pseudo-code, where it has to work for any values, and the JIT pseudo-code which already knows the values since it is compiled whilst the program is running:

Precompiled:
add(total, deposit)
subtract(total, withdrawal)

OR

add(total, subtract(deposit, withdrawal))

JIT:
add(total, 15)
subtract(total, 15)

OR

add(total, subtract(15, 15))

The JIT compiler can skip either of those, since it knows they cancel out each other's effects.

JIT compilation is becoming more and more common, with JIT compilers being written for Java and Smalltalk, for example. There is even JIT support in EUAE (the E(nhanced/xtended/ggplant) U(NIX/biquitous) Amiga Emulator, the exact naming of which is open to interpretation). An emulator translates programs written for one computer into something which will run on a different type of computer. In EUAE's case this means running code written for the chips in Amiga computers (such as the Motorola 68020) on different chips like the Intel 386. This used to be done by treating the Amiga programs like an interpreted language, with the emulator acting as the virtual machine. With the JIT engine, however, the Amiga programs can be run directly on the different chip, with the "compilation" actually being a translation from one chips instructions to another's.

A very promising project currently being worked on is the Low Level Virtual Machine (LLVM). Traditional compilers, like the GNU Compiler Collection, work by translating the given language into a common, internal language which the compiler can work with. Various optimisations can then be done on this internal representation before translating it into machine code for whatever computer is requested. LLVM, however, is slightly more clever. It performs the same translation and optimisation, but is not only able to translate this internal representation into machine code, it also has a virtual machine capable of interpreting it and is even able to JIT compile it, depending on the options chosen by the user. This means that, for example, C code, which is the classic example of a compiled language, can be fed into LLVM and made to run in a virtual machine or be JIT compiled. The same goes for Ada, Smalltalk and other languages which LLVM can handle. This means that LLVM could potentially (when bugs are fixed and GCC-specific assumptions in some programs are handled) make almost the whole of a computer system compile Just-In-Time and be optimised on-the-fly (not quite everything though, since something needs to start up the JIT compiler :P ). LLVM could even optimise itself by compiling itself. Likewise an entire system could be run in a virtual machine without the need for the likes of KVM or Virtual Box. The future looks pretty interesting!

** Computer processors can be as fast as you care to make them, but that doesn't matter if you can't give them things to do at the same rate. The processor contains "registers" which contain the numbers it's dealing with, however these registers can only contain one thing at a time and thus their values have to keep changing for each operation. The place where numbers can actually be stored over a long time is the RAM (Random Access Memory), the processor's registers can therefore get their next values from any point in the RAM and dump the results of calculations just performed into any other point in the RAM. However, this is a huge slowdown, so processors have a cache which is able to store a few things at a time, and the registers are connected to the cache rather than the RAM. This is a lot faster, since frequently used values can be stored in the cache and accessed really quickly (these are called "hits").However, if something needed is not yet in the cache then it has to be taken from the RAM (a "miss") which a) takes time and b) means kicking something else out of the cache to make room for the new value.

There are various algorithms for trying to keep cache misses as low as possible, but an obvious way is to use small numbers. In a really simplified (and decimal rather than binary) example, let's say a cache can store ten digits, then I can store for example 5, 16, 3, 78, 4, 67 and 1 all at once, since that is 10 digits in total. That can't really be made any more efficient, so if a different number is needed (a miss occurs) then at least one of them has to get replaced. If a program is being overly cautious about the size of the numbers it is expecting, then it might use the cache inefficiently. For example, say the program is anticipating values of a few hundred, it might want to store the number 547 which would require three places in the cache. If the number it actually deals with turns out to be five, then it will store 005 in the cache, wasting two spaces, replacing numbers it doesn't need to and thus increasing the likelyhood of a miss. If there is a miss later and one or two of those three digits are chosen to be replaced then the whole 005 gets kicked out rather than just the wasteful zeroes, meaning that any future calculations needing that 5 will result in a miss and the whole 005 will have to be brought in again from the RAM, causing the same inefficiencies again.

OK, back to revision :P