Thursday, 10 December 2020

 Google and Facebook paying media companies

In The Age newspaper, December 10, was an article about why Google and Facebook need to be charged by the Australian media companies in order to keep media reporting alive ("The world may follow our lead on digital news", Peter Lewis). While I am sympathetic to their plight, I regard the arguments as fundamentally flawed. I wrote a letter to the The Age saying this, but it was not published. Now that's fair enough, I only get about a 20% success rate in letters to the editor. But I have noticed that The Age often likes to publish pairs of letters, one for and one against any particular topic. Alas, today's paper only had only one letter presenting only one side, agreeing with the article.

With the Web and the rise of search engines, many business models have irrevocably changed. We work in different ways, purchase in different ways, and get information in different ways. This inevitably means that some styles of business will suffer while others will blossom. In all the media forms this has resulted in a shift of advertising money from the traditional media to the web companies. So does that mean money should be diverted back to support the media companies? Following through: should coal miners be subsidised because people are shifting to solar? Should Coursera and a host of online education companies be required to pay schools and unis for decreasing enrolments? Should Kogan and other online retail agents be required to subsidise the shopping malls with falling tenancies? It's not a strong argument!

I still get most of my news through the paper copy, or online when the paper isn't delivered. I'm a subscriber and go directly to the newspaper's web site. When I want to find out something I don't know, I go to a search engine. That may be for more titillating news about the mad US President, but is just as likely to be for reviews of, say, vacuum cleaners, or where I can buy a new lawnmower, or does a word exist that I want to put in the crossword. The search engines don't care what my query is, they just point me to sites with hopefully relevant content. Should the search results be treated differently just because they refer to "news"?

What the search engines give me are links to other web sites. True, Google vectors them through its own site for tracking purposes but that is a different issue. The site news.google.com doesn't have copies of the articles, just a headline and a link. That is very different to actually showing copies - that would be breaking copyright and rightfully punishable. When I am looking for lawnmowers, the search engines give me a link to Bunnings, and when I want a review they often point me to the Choice website. Again, why should "news" be treated differently?

Searching for up-to-date news on the US President, I often get links to US media companies. When I follow through, I'm often met with a subscriber paywall. That is totally valid: you want to see something that has cost money to produce, then you can be expected to pay for it. I also read news online from the New Daily website (https://thenewdaily.com.au/). Their site is littered with adverts. Again, a valid model. It is also supported by superannuation funds as are many companies by many organisations.

I run my own web site and put my professional work there: research papers, books, links to this blog and so on. Every now and then someone comes to my site through a search engine. Google doesn't pay me for my content, and I don't pay Google for indexing it. It's win-win for both of us. I didn't know about sites like news.google.com until the media started making a fuss about it, but that also looks like a win-win: Google sifts and categorises the news and presents links to follow through.

It is true that income to the media companies is suffering and the quality of news may fall as a result. It may be that subsidies of some kind are appropriate. The questions should be about who should be subsidising who, not just a blanket decision to target Google and Facebook. The New Daily is an example of one way. Murdoch's News Corp subsidies to the loss-making Foxtel are another. Let's have that conversation rather than a flawed piece of legislation.


Friday, 26 May 2017

More on Seyed Hossein Ahmadpanah


This guy is really good! Plagiarised from Niklaus Wirth even!
Here is Seyed's data structures book Seyed's book

And here is from Wirth's book at Algorithms and Data Structures



I should be honoured to be considered worth plagiarising in such company!




Plagiarism by Seyed Hossein Ahmadpanah

I've had an a book online for many years "Network Programming with Go"  https://jan.newmarch.name/golang. under a Creative Commons license which allows copies but does not allow change of copyright to another person. It is now being published by APress.

It was pointed out to me that Seyed Hossein Ahmadpanah has self-published a book "Go for Network" which is a complete ripoff of my book. Well, he did change one thing: the name of the author from me to him! Checking up, this guy is really prolific: according to ISBN.nu he has published 24 books!

A list of his magnificent achievements is given below. I haven't checked many of them, but in addition to mine, the Start Linux book appears to be stolen from Alex Carr Dive into Linux
If you know the original authors of any of these books, please let them know!

Data Structure

 The modern digital computer was invented and intended as a device that should facilitate and speed up complicated and time-consuming computations. In the majority of applications its capability to store and access large amounts of information plays the dominant part and is considered to be its primary characteristic, and its ability to compute, i...read more
By Seyed Hossein Ahmadpanah

Learn CSS3

 If you've been a web developer for any length of time, then CSS won't be strange to you. You might, however, be wondering what the fuss over CSS3 is all about. Just like the new HTML 5 specifications, CSS3 (or CSS Version 3, to be more precise) is the latest set of specifications designed to mold, shape, and define just what capabilities the newest version of CSS has...read more
By Seyed Hossein Ahmadpanah

Go for Network

 You can't build a system without some idea of what you want to build. And you can't build it if you don't know the environment in which it will work. GUI programs are different to batch processing programs; games programs are different to business programs; and distributed programs are different to standalone programs...read more
By Seyed Hossein Ahmadpanah

How to Start Java Programming

Why another book on Java? Why a book on Java and Linux? Isn’t Java a platform-independent system? Aren’t there enough books on Java? Can’t I learn everything I need to know from the Web?
By Seyed Hossein Ahmadpanah

How Operating Systems Work

 All desktop computers have operating systems. The most common are the Windows family of operating systems developed by Microsoft, the Macintosh operating systems developed by Apple and the UNIX family of operating systems (which have been developed by a whole history of individuals, corporations and collaborators)...read more
By Seyed Hossein Ahmadpanah

Proactive Cyber Defense: Security for Government

Proactive cyber defense in Government relates to the way a company is run and managed in order to ensure its well-being. All of a company’s stakeholders are basically players as far as good Government is concerned, but the responsibility and accountability for it starts with its Board of Directors and Senior Management...read more
By Seyed Hossein Ahmadpanah

Data Mining for Big Data

The most commonly accepted definition of “data mining” is the discovery of “models” for data. A “model,” however, can be one of several things. We mention below the most important directions in modeling.
By Seyed Hossein Ahmadpanah

Why Unit Test

By Seyed Hossein Ahmadpanah

Start Wordpress

Start WordPress Right Now !
By Seyed Hossein Ahmadpanah

Start Linux

Linux is a Unix-like open source operating system. At the core of the operating system is the Linux kernel. It acts as the intermediary between the applications which run in the operating system and the underlying hardware.https://www.amazon.com/Dive-into-Linux-Alex-Carr-ebook/dp/B01FFFRBBK
By Seyed Hossein Ahmadpanah

Start Php

 PHP is a powerful programming language that lets you build dynamic Web sites. It works well on a variety of platforms, and it's reasonably easy to understand.
By Seyed Hossein Ahmadpanah

Start Bootstrap

 Bootstrap, quite simply, puts the power of design and UI layout back in the developer’s hands, leaving things in a nice, clean, easy-to-maintain state that can then be passed to a front-end graphic designer whose job it is to make things shine.
By Seyed Hossein Ahmadpanah

What Is Git ?!

Git is an open-source version control system known for its speed, stability, and distributed collaboration model. Originally created in 2006 to manage the entire Linux kernel, Git now boasts a comprehensive feature set, an active development team, and several free hosting communities...read more
By Seyed Hossein Ahmadpanah

What Is Angular.js?! 

Angular.js is an open-source JavaScript framework developed by Google. It gives JavaScript developers a highly-structured approach to developing rich, browser-based applications which leads to very high productivity. If you are using Angular...read more
By Seyed Hossein Ahmadpanah

Full Android Guide: Advanced

 Android has grown from nothing to arguably the world’s most popular smartphone OS in a few short years. Whether you are developing applications for the public, for your business or organization, or are just experimenting on your own, I think you will find Android to be an exciting and challenging area for exploration...read more
By Seyed Hossein Ahmadpanah

Start Mongodb

Most of this book will focus on core MongoDB functionality. We’ll therefore rely on the MongoDB shell. While the shell is useful to learn as well as being a useful administrative tool, your code will use a MongoDB driver. This does bring up the first thing you should know about MongoDB: its drivers...read more
By Seyed Hossein Ahmadpanah

Full Android Guide: Intermediate

By Seyed Hossein Ahmadpanah

Full Android Guide: Beginner

By Seyed Hossein Ahmadpanah

Start Node.js

Node - or Node.js, as it is called to distinguish it from other "nodes" - is an event-driven I/O framework for the V8 JavaScript engine. Node.js allows Javascript to be executed on the server side, and it uses the wicked fast V8 Javascript engine which was developed by Google for the Chrome browser...read more
By Seyed Hossein Ahmadpanah

Start SQLNetwork

This book will cover the basics of SQL, and should help introduce beginners to SQL concepts. It is also a good documentation source when forgetting how the basic SQL syntax works.
By Seyed Hossein Ahmadpanah

Start Js!

JavaScript (JS for short) is the programming language that enables web pages to respond to user interaction beyond the basic level. It was created in 1995, and is today one of the most famous and used programming languages.
By Seyed Hossein Ahmadpanah

Start Go!

Go is an open source programming language that makes it easy to build simple, reliable, and efficient software.
By Seyed Hossein Ahmadpanah

Start Python 3!

This is a simple book to learn Python programming language, it is for the programmers who are new to Python.
By Seyed Hossein Ahmadpanah

Start Ruby !

 Start Ruby! will help you begin programming in Ruby
By Seyed Hossein Ahmadpanah

Friday, 22 May 2015

Linux graphics on Gigabyte Brix i7 5500

I've been installing and running Linux on sooooo many computers for years and it hasn't been a problem for sooooo many years. I thought it was easy now :-(. The Gigabyte Brix GB-BXi7H-5500 has got me. (But in the end I won). The problem is crazy graphics display when trying to use any of the desktop live installs, making Linux apparently un-installable.

It only has HDMI output and mini DisplayPort. I started with Ubuntu 14.04. Ho hum, the HDMI driver didn't work and I just got garbage on the screen DURING THE INSTALL PROCESS. Same with some other distros: if they wanted to use graphics with the HDMI card then I couldn't get going. And most distros want graphics. Don't confuse this with GUI options once you have a distro installed: all distros will allow a runtime non-GUI interface. I needed one at the INSTALL stage!

So, a server only distro might give that. I tried copying servers to a USB stick using unetbootin which previously has worked great. But I tried Ubuntu server and Fedora server and bizarrely neither of them worked: they kept failing to find necessary pkg files from the download servers. Eventually I got Ubuntu server to install - I guess the crappy Australian internet caused timeouts previously.

Booting off a USB stick wasn't fun. You have to turn off UEFI security in the BIOS boot menu (keep bashing the Del key on bootup to get to it). Even then, in the Boot Selection menu it only showed the USB stick as a UEFI boot, and that wouldn't boot. But in the Save and Exit menu it does show as both UEFI and non-UEFI boot. Choose the non-UEFI option.

For partitioning the hard disk, the default do-it-all-for-you only gives 8Gb for the root partition. That includes /var and /tmp so is ridiculously small. I prefer to set the disk partitions myself, giving 30Gb to root.

Then you can install the server - almost. Eventually you get to the Install Grub to a Disk. If you try it, it will fail trying to write to your USB stick on /dev/sda instead of your hard disk /dev/sdb :-(. If you don't try it, the grub packages won't be installed which will cause problems later. Either way, you have to end up selecting Don't install a boot loader.

Without a boot loader, the new server won't boot of course! Instead, using the USB again you need to get into Rescue mode. That sets keyboards and locations, downloads a base system, etc. Eventually you will get asked to drop into a shell prompt. Choose one for your hard disk's root partition. If you tried and failed to install grub to your boot sector then at least the grub package will be there. Otherwise you have to download and install the grub package - see next para.

apt-get is what you use to download packages. It requires networking to be okay. In rescue mode the resolve file /etc/resolv.conf is empty. It links to a non-existent file, so remove it first. Then you can edit it to add
     nameserver 8.8.8.8
(Google's nameserver) or similar. Then you can run
    apt-get install grub

Once the grub package is there, install it by
   grub-install /dev/sdb
   update-grub
(assuming your target hard disk is currently mounted as /dev/sdb).

So now I have a server system installed. To see if X worked, I ran
   sudo apt-get install xinit
and yes, it did. You can start X by
   startx
and it will run a single xterm. I then installed Tom's Window Manager twm. It's really basic but that's okay. I can run that from within my xterm to give me some control. i had to edit my .xinitrc to include
   #!/bin/bash
   xterm &
   twm    # synchronous!
Note that the display is still broken, but doing something really simple like this works (just).

So I got more ambitious and ran
   sudo apt-get install gnome-shell
That took over the login process and trashed my screen again, just like the install process. So there is my problem showing up.

By ctl-alt-F4 I can get into a terminal screen and check things out. I can run basic X, and it looks odd but something works. I installed Firefox but when I ran it, just a mess again - too complex for the problem case.

Maybe an up-to-date kernel. I downloaded kernel sources for 4.0.2 to another machine (no working browser on this one!) and copied it across. Built and installed it
   make
   make install
   make modules_install
and it wouldn't boot as it couldn't find my hard disk. Also needed
   update-initramfs -u
So it booted, but the problem with the display didn't go away. Kernel 4.0.2 wasn't the answer. Was it the monitor? No, worked okay with another computer (well, sort of okay - the mouse left tracks across the screen, but I've seen that problem elsewhere with the HDMI driver. Kernel 4.0.2 was not the solution.

With a working command-line Linux at least I can now find out things about the Gigabyte. The cpu is an Intel i7-5500U @ 2.4 Ghz (but I knew that already from the specs). The framebuffer is an inteldrmfb. The graphics driver is the Intel i915.

The site
https://01.org/linuxgraphics/documentation/how-report-bugs
seems to be all about Intel graphics drivers. From there I found the Graphics Installer 1.0.8 for Ubuntu* 14.10, 64-bit
The fun starts here again. It requires Ubuntu 14.10. The defaults on the Ubuntu site are 14.04 and 15.04 so you need Google to find the server 14.10. Then it installs just as described above. The graphics installer has to run under X and apart from a few glitchy effects is navigable. But once it gets going, lots of flashing lights, stray bolts of lightning, etc from the graphics card. I switched to another virtual terminal until it finished. But it made no difference - no usable X. Not a solution.

Time to look for bug reports. I found the bug "Graphics unstable on Ubuntu 14.04 and 14.10 using Intel HD Graphics 5500" at https://bugs.launchpad.net/xserver-xorg-video-intel/+bug/1432194
Fix #36 starts the X server in UXA mode instead of the preferred SNA mode (no, I don't know what they are)

    try to create the following folder:
    sudo mkdir /etc/X11/xorg.conf.d/

    Then create inside the new folder a file named for example "20-intel.conf" with the following content:
      Section "Device"
         Identifier "Card0"
         Driver "Intel"
         Option "AccelMethod" "uxa"
         EndSection

    Then reboot and voila."

This works! Most of the time.  Fix #100 didn't work on Ubuntu 14.10 (unresolved packages) although reports were that it worked for 14.04. But the moderator for that bug said that my bug wasn't his bug and I should log a new one.

In the meantime I had logged bug 90401 at bugs.freedesktop.org. Comment #5 pointed me to the development site for the xf86 intel video driver

   Please test the Xorg ddx driver from [1] and report back.

   [1] http://cgit.freedesktop.org/xorg/driver/xf86-video-intel/

I downloaded 2.99.917commit baec802b21... and so far that one is okay. To build it required installing packages autoconf, autogen, autoreconf, xutils-dev xorg-x11-server-devel, libtool, pkg-config. Then run
    ./autogen.sh
    make
    make install
in the xf86-video-intel directory

This removes /usr/lib/xorg/modules.drivers/intel_drv.so and installs /usr/local/lib/xorg/modules.drivers/intel_drv.so. So far I can run Big Buck Bunny, the test suite glmark2 (test result only scores 119) (I got info about this test from Benchmark graphics card (GPU) performance on Linux with glmark by Silver Moon), browsers with YouTube movies etc.

So finally X is working, but I only have the server install. Get the GUI desktop by
  sudo apt-get update
  sudo apt-get install ubuntu-desktop

This  gives the Unity desktop. I'm not a fan of Unity, so I usually install old-style gnome by
  sudo apt-get install gnome-session-fallback
and Compiz (for wobbly windows) by
  sudo apt-get install compiz

Summary
1) install Linux using a server package with a command-line install. 
2) Get the latest source version of xf86-video-intel, build and install it
3) X should now work if you want to test it by downloading e.g. xinit and twm
4) Install the desktop package ubuntu-desktop






     



Monday, 29 December 2014

Brandis caught out by metadata

George Brandis, the Attorney General for Australia is proposing a compulsory metadata retention scheme for Australian ISPs. He showed on a TV interview a few months ago that he basically had no idea what metadata was all about, mumbling about addresses on (snail mail) envelopes. Well, this week it should have been brought home to him what the fuss is all about.

Brandis hosted a dinner party in London on April 4 for "a group of senior British arts representatives". The bill was £627, with £228 in alcohol charges alone. The Age newspaper reported this as "George Brandis' 'obscene' $1100 dinner funded by taxpayers".

The food and wine were the "data" in this transaction. The topics of conversation could probably be called data too.  The cost, and the people involved would be the metadata. We don't know what was discussed, but we do know that the metadata itself has left a big stain on Brandis' character and judgement. So George, metadata counts, and your proposed legislation needs to be very careful about what is collected and who can use it.

Wednesday, 17 April 2013

Behringer MIX800 ultra compact Karaoke machine

My search for Karaoke on my laptop led me to writing to a book still in alpha on Linux sound programming. But I kept returning to the original topic, and my question this time was on how to support two microphones. I also wanted to introduce effects like reverb because it makes the voice sound richer. I suppose that means two sound cards unless I can find one with two microphone inputs. Basically, to do it in software I need to duplicate a mixer preferably with FX for reverb.

Real mixers aren't usually cheap. The ones in Chinese DVD players are though, as a complete player with two microphone inputs and reverb effects is often less than $100. But I had thrown out my old players :-(.

Then I came across the Behringer MIX800 ultra compact Karaoke machine. You can get it from Amazon for US$64. I paid A$109 from an Australian distributor Store DJ. It's good: two microphone inputs with reverb/echo. One line input from your CD/Computer with voice cancellation (which works sort of). Play the karaoke files on your computer using kmid or my Java programs, feed the output to the MIX800, plug in your microphones and send the mixed output to an amp. Easy! No need to mess around with mixing by PulseAudio or other s/w. I'll get the s/w working eventually as I want it, but for now this is a very good and easy solution.

Wednesday, 30 May 2012

Found low latency for Karaoke with PulseAudio

As a followup to the last post: I can now play Midi files and sing along with no noticeable latency using PulseAudio. To direct the (default) microphone to the (default) speaker load the module_loopback (thanks to rusty0101):

    pactl load-module module-loopback latency_msec=1

Everything you speak/sing/holler will then be played on the speaker.

For Midi file playback I discovered FluidSynth. The following program will play a Midi file:

// See http://fluidsynth.sourceforge.net/api/

// Run by ./PlayFile /usr/share/soundfonts/FluidR3_GM.sf2 ../54150.mid

int main(int argc, char** argv)
{
    int i;
    fluid_settings_t* settings;
    fluid_synth_t* synth;
    fluid_player_t* player;
    fluid_audio_driver_t* adriver;
    settings = new_fluid_settings();
    synth = new_fluid_synth(settings);
    player = new_fluid_player(synth);

    fluid_settings_setstr(settings, "audio.driver", "pulseaudio");
    // Use paman to find output device's name
    fluid_settings_setstr(settings, "audio.pulseaudio.device",
              "alsa_output.pci-0000_00_1b.0.analog-stereo");

    adriver = new_fluid_audio_driver(settings, synth);
    /* process command line arguments */
    for (i = 1; i < argc; i++) {
        if (fluid_is_soundfont(argv[i])) {
           fluid_synth_sfload(synth, argv[1], 1);
        }
        if (fluid_is_midifile(argv[i])) {
            fluid_player_add(player, argv[i]);
        }
    }
    /* play the midi files, if any */
    fluid_player_play(player);
    /* wait for playback termination */
    fluid_player_join(player);
    /* cleanup */
    delete_fluid_audio_driver(adriver);
    delete_fluid_player(player);
    delete_fluid_synth(synth);
    delete_fluid_settings(settings);
    return 0;
}


And whadda-you-know? It all works fine.

There's just the matter of hooking up a GUI, and a few thousand lines of code. But at least I'm starting from a good base, and I now realise the Java Sound framework can't give me that, sad to say.

Tuesday, 29 May 2012

In search of (low) latency

This is a followup to my investigations into playing my Songken DVD DKD files on my laptop. In an earlier blog I described how to decode the DKD files into Midi or Midi+WMA files. The intent was then to build a Midi player that would also show the notes of the melody and also the notes the singer was singing.

Well, I did all that. Java Sound has a Midi player. Java Sound has a Sampled API to handle sounds from the microphone to the loudspeaker. Java has a GUI for showing stuff. TarsosDSP by Joren Six has implemented a number of pitch detection algorithms such as YIN and they can be pulled in to give an estimate of the pitch sung. Java can convert characters from language encodings such as GB2312 to Unicode and display them so I can see Chinese and other characters.  So it's all there....

... but latency still kills it. The Midi player introduces latency somehow into the sampled sounds, but even if you work around it - even if you just do sampled data alone - then there is still that little delay. Here are my Java source files. Maybe I will write up an explanation of what I was doing with them later. I'm going to stop work on them right now till I get the latency sorted out.

The standard audio system for (consumer) sound on Linux is Pulse Audio. But as Lennart Poettering explained at the Linux Audio Conference 2010, pro audio has different aims to consumer audio, and this project is closer to pro audio than consumer audio (although to think of Karaoke singers as pros is stretching it a bit :-). In consumer audio, latencies of upto 2 seconds may be permissible, while pro audio sets an upper limit of 20 milli-seconds.

Java Sound is estimated to have a 50msec delay: "These measurements suggest that the latency introduced by buffers in the "Java Sound Audio Engine" is about 50 ms, independant of the sample rate." Now that's on old equipment, but it means there is an uphill struggle.

The sound quality of the builtin soundcard HDA Intel PCH (STAC92xx) on my Dell laptop is appalling. That has to be overcome too. This laptop doesn't have a microphone input, so I started looking at USB sound cards. My first attempt was with a AnPu Portable USB 3D Virtual 5.1 Audio Sound Card Adapter Blue  from Dino Direct. Dino was good: delivery post-free within 2 weeks. But the card was cheap (A$4) and broke when I inadvertently yanked it out of the USB slot.


My second attempt was with Swamp Industries for an XLR to USB Adapter. That was about A$20 but I got it with a microphone as well. The service was good again. Well, the card's okay for input, but still has to go out through the onboard soundcard.


The third attempt was with a Sound Blaster X-Fi Surround 5.1 Pro at A$70. It's a USB 1.1 device (Linux still has issues with USB 2 devices, apparently).  Pulse Audio only recognises it as an input device, not as an output device, so it didn't seem to improve things.


Pulse Audio is an audio layer above Alsa (OSS was used previously to Alsa). Alsa could see the device fine:

$arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: STAC92xx Analog [STAC92xx Analog]
  Subdevices: 0/1
  Subdevice #0: subdevice #0
card 2: Pro [SB X-Fi Surround 5.1 Pro], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
 

and

$aplay -l
...
card 2: Pro [SB X-Fi Surround 5.1 Pro], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: Pro [SB X-Fi Surround 5.1 Pro], device 1: USB Audio [USB Audio #1]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
 

Now about this time I went off on what turned out to be a wild goose chase (at least so far) by looking at Jack: "JACK is [a] system for handling real-time, low latency audio (and MIDI)". Jack currently also uses Alsa. Now that looks good - but Java Sound and Jack don't play together.

Java Sound has a couple of weird bits where "obviously equivalent" things aren't. I hit this first with volume control in playing a Midi file: you can't set the volume on the default device but you can if you iterate through the devices and select the default one. Then you can set the volume on it. Huh? Thanks to Greg Donahue for solving that one. You hit similar problems trying to find the sound cards and you end up either with
  • Java Sound not playing to your default card; or
  • When you explicitly select the default card then Java Sound throws an exception saying that its PulseAudio drivers can't find it.
So after all that, where are we?
  • Java Sound has latency problems
  • The inbuilt soundcard is crap
  • Pulse Audio can't properly find the USB soundcard
  • Java Sound uses Pulse Audio
  • Pulse Audio has latency issues
  • Jack is ignored by Java Sound
  • Alsa and Jack can find the USB soundcards
Is it possible to have latency-free sound on Linux? Well, Jack claims to be latency-free, but then it has to go through the Alsa layer. Can the Alsa layer be latency-free? Not completely, but I finally figured out the following test:

    arecord  -f dat -B 4  -D hw:0| aplay -B 4 -D hw:2 -f dat -

i.e record at DAT standard (16 bits, 48k samples) from the builtin mike (hw:0) played on the USB soundcard (hw:2), with 4msec buffer time. And hey! It works! No latency that my poor ear can hear. This simple pipeline isn't perfect: any overrun introduces latency into the pipeline, but that can be handled in code by dropping samples. The sample size can be increased and it still sounds okay - 4ms was the lowest I could take it.

Conclusion:
  • the top-down approach through Java works but has latency issues
  • the bottom-up approach through Alsa handles latency
I just need to combine the two...



Saturday, 5 May 2012

java sound: midi and sampled streams played together

I've been playing around with my home Karaoke system. This has included decoding my Songken DKD disk. Once I did that, I wanted to emulate the behaviour of my Malata Karaoke player: playing the Midi files, showing the lyrics and also showing in a bar graph the notes that should be sung and the notes the performer is actually singing.

This requires processing files of Midi data, handling the soundcard microphone input and speaker output and using a GUI to show everything. Java Sound looks like a perfect choice for this as it can do all these things. (Although Oracle's custodianship of Java and their outrageous API copyright claims makes it increasingly difficult to justify starting a new project using Java.)

Playing a file of Midi data is easy:

    try {
        Sequence sequence = MidiSystem.getSequence(midiFile);
        Sequencer sequencer = MidiSystem.getSequencer();
        sequencer.open();
        sequencer.setSequence(sequence);
        sequencer.start();
    } catch (Exception e) {...}

Copying sound from the microphone to the speaker is a bit harder. You have to set up TargetDataLine to read bytes from the microphone, set up a SourceDataLine to send bytes to the speaker and then copy bytes from the target to the source (yes, that's the correct way though the nomenclature is strange, copying from the target of the input mixer to the source of the output mixer) [based on code by  Matthias Pfisterer]:

    private static AudioFormat getAudioFormat(){
        float sampleRate = 44100.0F;
        //8000,11025,16000,22050,44100
        int sampleSizeInBits = 16;
        //8,16
        int channels = 1;
        //1,2
        boolean signed = true;
        //true,false
        boolean bigEndian = false;
        //true,false
        return new AudioFormat(sampleRate,
                   sampleSizeInBits,
                   channels,
                   signed,
                   bigEndian);
    }//end getAudioFormat

    public  void playAudio() throws Exception {
        AudioFormat audioFormat;
        TargetDataLine targetDataLine;
   
        audioFormat = getAudioFormat();
        DataLine.Info dataLineInfo =
            new DataLine.Info(
                  TargetDataLine.class,
                  audioFormat);
        targetDataLine = (TargetDataLine)
            AudioSystem.getLine(dataLineInfo);
   
        targetDataLine.open(audioFormat,
                audioFormat.getFrameSize() * FRAMES_PER_BUFFER);
        targetDataLine.start();
   
        playAudioStream(new AudioInputStream(targetDataLine));
    } // playAudioFile
    
    /** Plays audio from the given audio input stream. */
    public  void playAudioStream( AudioInputStream audioInputStream ) {
        // Audio format provides information like sample rate, size, channels.
        AudioFormat audioFormat = audioInputStream.getFormat();
    
        // Open a data line to play our type of sampled audio.
        // Use SourceDataLine for play and TargetDataLine for record.
        DataLine.Info info = new DataLine.Info( SourceDataLine.class,
             audioFormat );
        if ( !AudioSystem.isLineSupported( info ) ) {
            System.out.println( "Play.playAudioStream does not handle this type of audio on this system." );
            return;
        }
    
        try {
             SourceDataLine dataLine = (SourceDataLine) AudioSystem.getLine( info );

            dataLine.open( audioFormat,
               audioFormat.getFrameSize() * FRAMES_PER_BUFFER);
        
            // Allows the line to move data in and out to a port.
            dataLine.start();
    
            // Create a buffer for moving data from the audio stream to the line.
            int bufferSize = (int) audioFormat.getSampleRate() *
            audioFormat.getFrameSize();
            bufferSize =  audioFormat.getFrameSize() * FRAMES_PER_BUFFER;
            // See http://docs.oracle.com/javase/6/docs/technotes/guides/sound/programmer_guide/chapter5.html
            // for recommendation about buffer size
            byte [] buffer = new byte[bufferSize / 5];
    
            // Move the data until done or there is an error.
            try {
                int bytesRead = 0;
                while ( bytesRead >= 0 ) {
                    bytesRead = audioInputStream.read( buffer, 0, buffer.length );
                    if ( bytesRead >= 0 ) {
                        int framesWritten = dataLine.write( buffer, 0, bytesRead );
                    }
                } // while
            } catch ( IOException e ) {
                e.printStackTrace();
            }
            dataLine.drain();
    

            dataLine.close();
        } catch ( LineUnavailableException e ) {
            e.printStackTrace();
        }
    } // playAudioStream

Now both of those work okay, picking up default devices, mixers, data lines, etc. You have to be careful running the copy code from microphone to speaker - you can set up a howling feedback loop between your laptop's microphone and speaker if you don't use, say, headphones.

There is a detectable latency (delay between the sounds) between talking/singing into the microphone and getting sound out of the speaker, but it is acceptable. But when you put the two pieces of code in the same program - even in different threads - then the latency blows out and the result isn't acceptable after all. There is a distinct delay between the input and the output sounds. Processing the Midi data somehow interferes with processing the sampled data and introduces additional delays which make it unusable.

I looked around on the Web, and read all the Sun/Oracle documentation that I could find, but couldn't find anything talking about this problem in the context of the Java Sound API. I've now found a solution (even if it isn't totally portable) so that is why I'm writing this blog.

The above code leaves almost everything to defaults. So the Midi code must be re-setting some default used by the sampled data code. The most likely candidate is the output Mixer, but you can't get from the SourceDataLine to its Mixer, and the Midi API nowhere gives you access to things like Mixers. Digging around in the OpenJDK source code showed lots of interesting things such as the Midi code setting its Midi-processing thread loop to a very high priority but I ran out of steam before finding the link between the two processing streams. The com.sun.media.sound package has a bunch of software mixers - the answer is probably in there somewhere.

So I looked at the mixers available. The following function shows how:

  public void listMixers() {
    try{
        Mixer.Info[] mixerInfo =
            AudioSystem.getMixerInfo();
        System.out.println("Available mixers:");
        for(int cnt = 0; cnt < mixerInfo.length; cnt++){
            System.out.println(mixerInfo[cnt].getName());  
        }//end for loop
     } catch(Exception e) {
     }
  }
On my laptop running Fedora 16 this lists

Available mixers:
PulseAudio Mixer
default [default]
PCH [plughw:0,0]
NVidia [plughw:1,3]
NVidia [plughw:1,7]
NVidia [plughw:1,8]
Port PCH [hw:0]
Port NVidia [hw:1]

There's a default mixer which I don't want, several hardware mixers and a PulseAudio one. PulseAudio is the audio system on most current Linux systems so I get the best (Linux) portability by choosing that one, while avoiding whatever default Java Sound gives me.

Do these mixers support source lines and target lines? This shows the full list

            System.out.println("Available mixers:");
            for(int cnt = 0; cnt < mixerInfo.length;
                cnt++){
                System.out.println(mixerInfo[cnt].
                                   getName());
               
                Mixer mixer = AudioSystem.getMixer(mixerInfo[cnt]);
                Line.Info[] sourceLines = mixer.getSourceLineInfo();
                for (Line.Info s: sourceLines) {
                    System.out.println("  Source line: " + s.toString());
                }
                Line.Info[] targetLines = mixer.getTargetLineInfo();
                for (Line.Info t: targetLines) {
                    System.out.println("  Target line: " + t.toString());
                } 
            }//end for loop

This shows results like
  PulseAudio Mixer
    Source line: interface SourceDataLine supporting 42 audio formats, and buffers of 0 to 1000000 bytes
    Source line: interface Clip supporting 42 audio formats, and buffers of 0 to 1000000 bytes
    Target line: interface TargetDataLine supporting 42 audio formats, and buffers of 0 to 1000000 bytes

(Note that you have to ask for mixer.getSourceLineInfo() - asking for mixer.getSourceLines() only shows the open lines and there will be none of those till you open them!)

I leave the Midi code alone. I don't need to mess with it. The sampled data I handle this way:

            Mixer.Info[] mixerInfo = AudioSystem.getMixerInfo();
            Mixer mixer = null;
            for(int cnt = 0; cnt < mixerInfo.length; cnt++){
                if (mixerInfo[cnt].getName().equals("PulseAudio Mixer")) {
                    mixer = AudioSystem.getMixer(mixerInfo[cnt]);
                    break;
                }
            }//end for loop
            if (mixer == null) {
                System.out.println("can't find a PulseAudio mixer");
            } else {
                Line.Info[] lines = mixer.getSourceLineInfo();
                if (lines.length >= 1) {
                    try {
                        dataLine = (SourceDataLine) AudioSystem.getLine(lines[0]);
                        System.out.println("Got a Pulse Audio source line");
                    } catch(Exception e) {
                    }
                } else {
                    System.out.println("no source lines for this mixer " +
                                                     mixer.toString());
                }
            }

And that's it! I can now write to this SourceDataLine and my sampled data is going straight to the Linux sound mixer, bypassing whatever the Java Sound Midi system is doing. Latency problem solved.

Now on to the next steps...

Sunday, 15 January 2012

Streaming audio and Android

n my home audio setup, I have all the music files in OGG format on my web server with an M3U playlist for each CD. This works fine on all my PC browsers. But almost none of it works for Android 2.2.Here is the stuff I tried and eventually got working for a Kogan Android TV Portal.

Saturday, 14 May 2011

REST, HTML, Ajax and PHP

What is REST?

REST is an architectural style for the Web. The term was coined by Roy Fielding in his PhD thesis. Roy was one of the architects of HTTP/1.1 so he knows what he was talking about. His thesis was rather abstract, so his ideas were ignored by the many who went chasing after SOAP, WSDL and UDDI in the hope that salvation lay that way. It didn't of course, and I have written about that in A Critique of Web Services.

REST as applied to the Web is based around the HTTP verbs of GET, PUT, DELETE and POST. Requests from a user-agent (such as a browser or web client application) signal what type of action they expect from a Web server by the choice of verb in the HTTP request. In a way, this is reminiscent of O/O programming where you get (GET) or set (PUT) a field of an object, delete (DELETE) an object or perform some arbitrary action (POST). I have explored this in An Overview of REST.

REST has become more popular as an approach to designing web applications, and has influenced many well-known systems such as Twitter and Google maps. Ruby on Rails is said to have adopted REST as a backend architecture. But the problem with an architecture is that it doesn't tell you how to do things, just what they should look like.

So in this blog, I'm going to look at building a simple web application from the front- to the back- end. It's a simple one: a database of people where each person has a name and ID, and you can query the database or add or delete from it.

From the REST viewpoint, a client will make these queries of the backend. The meanings of these are essentially given  by the HTTP specification:
HTTP method URI result
GET /people GETs a collection of people
POST /people POST: does something to the collection (using data in the POST request)
GET /people/jan GETs jan
PUT /people/jan PUTs a new version or an updated version of jan (using data in the PUT request)
DELETE /people/jan DELETEs jan
POST /people/jan POST: does something to "jan" (using data in the POST request)

The client will make these requests to an HTTP server. It will perform appropriate actions and return a result. The HTTP specification also specifies the result types:
HTTP method Status
GET 200 - found resource
404 - not found
POST 200 - resource returned
204 - no resource to return
201 - new resource created
PUT 201 - resource created
200 or 204- resource modified
4XX, etc - error occurred
DELETE 200 - deleted and resource returned
204 - deleted but no resource returned
202 - will be deleted
4XX, etc - won't be deleted

Python client
 
First I will give Python code to send these requests and interpret the responses. It is just a simple command-line interface, but you could wrap it in a GUI such as PythonCard.

#!/usr/bin/python

import httplib

baseURL = "/boxhill/ict329/webservices/people/"

def connect(method, url, data):
    connection = httplib.HTTPConnection("localhost")
    connection.request(method, url, data)
    response = connection.getresponse()
    status = response.status
    body = response.read()
    return (status, body)


def get(name):
    (status, body) = connect("GET", 
                              baseURL + name, 
                             "")
    if (status == 404):
        print name + ": no such person"
    else:
        print body

def put(name, ID):
    (status, body) = connect("PUT",
                             baseURL + name,
                             ID)
    if (status == 201):
        print name + ": created"
    else:
        print name + ": modified"

def delete(name):
    (status, body) = connect("DELETE",
                             baseURL + name,
                             "")
    if (status == 404):
        print name + ": couldn't delete"
    else:
        print name + ": deleted"
        print body

def getAll():
    (status, body) = connect("GET",
            baseURL,
            "")
    if (status == 404):
        print "No people"
    else:
        print body
 
You can run this with requests such as

get("Peter")
put("Fred", 21)
delete("Paul")

The backend will act on these requests and return suitable responses.

HTML and REST

The Web isn't just HTTP of course. HTTP is there to carry content, and most of the content is HTML (I'm talking value of content here, not size of content as in PowerPoint slides :-). How does HTML co-operate with REST?

HTML allows you to create documents with links, using tags such as <a href="...">. In the case of http: links, all browsers use an HTTP GET, although I can't find this specified anywhere.

HTML also allows you to create forms where the content can be submitted to a server. The type of submission is controlled by the "action" attribute of the form tag, and can be either POST or GET. This is true of HTML 4 and also currently true of the HTML 5 draft. I do have a copy of an 2009 draft which did allow PUT and DELETE, but now in 2011 those options have disappeared.

So HTML offers no support for PUT or DELETE and so cannot be considered supportive of REST. Using POST to mean GET, PUT, DELETE as well is not support as I would consider it.

JavaScript and REST

With JavaScript you can load() a document. This uses HTTP GET. You can submit a form by form.submit(). This uses the form's action attribute, which is either POST or GET.

"Standard" JavaScript does not have support for REST.

Ajax and REST

You have to turn to Ajax to get proper support for REST. That means you have to make Ajax calls using XMLHttpRequest if you want a browser to make the correct REST calls, whether you like it or not.

 I'll use JQuery as that is a little easier and takes care of browser differences compared to straight JavaScript. There is an object $.ajax which takes a dictionary of attribute: action pairs. This can be used to specify the calling method (GET/PUT/...) as well as the URL, and the action to take when a result is returned. For our purposes here, the HTTP return codes are 2XX Okay codes or 4XX error codes. These can be handled by $.ajax success: and error: elements respectively.

A function to GET a value and display the results or an error in an Alert box is

      function doGet() {
        name =  document.getElementById("get_name").value;
        $.ajax({
           type: "GET",
           url: "people/" + name,
           success: function(data, status, jqXHR) {
                      alert(data);
                },
           error: function(jqXHR, textStatus, errorThrown) {
                      alert(name + ": no such person");
                },
           async: false,
        });
      };
which extracts the name from a textbox with id "get_name" in a form and makes a synchronous GET request, and shows the appropriate alert box when the call returns.

A similar function can be used for the DELETE call:

      function doDelete() {
        name =  document.getElementById("delete_name").value;
        $.ajax({
           type: "DELETE",
           url: "people/" + name,
           error: function() {
                         alert(name + ": couldn't delete");
                  },
           success: function() {
                         alert(name + ": deleted");
                  },                     
           async: false,
        });
      };

For the PUT method, we want to distinguish between a response of 201 (the resource was created) and 204 (the resource was modified). Both of these are success values, but done in a different way. From JQuery 1.5 onwards, we can examine the status code of the HTTP response:

      function doPut() {
        name =  document.getElementById("put_name").value;
        ID =  document.getElementById("put_id").value;
        $.ajax({
           type: "PUT",
           url: "people/" + name,
           data: ID,
           statusCode: {
                201: function(jqXHR, textStatus, errorThrown) {
                         alert(name + ": created ");
                     },
                204: function(jqXHR, textStatus, errorThrown)  {
                         alert(name + ": modified");
                     }
           },
           async: false,
        });
      };
Finally, we link these calls to forms:
<ul>
  <li>
    <p>
      Get info about one person
    </p>
    <p>
      <form>
        Name <input type="text" id="get_name"/>
        <br/>
        <input type="button" value="Submit this"
                    onClick="doGet()"/>
       </form>
    </p>
  </li>

  <li>
    <p>
      Create a person
    </p>
    <p>
      <form>
        Name <input type="text" id="put_name"/>
        <br/>
        ID <input type="text" id="put_id"/>
        <br/>
        <input type="button" value="Submit this"
                    onClick="doPut()"/>
      </form>
    </p>
  </li>

  <li>
    <p>
      Delete a person
    </p>
    <p>
      <form>
        Name <input type="text" id="delete_name"/>
        <br/>
        <input type="button" value="Submit this"
                    onClick="doDelete()"/>
      </form>
    </p>
  </li>
</ul>

Apache and REST

That's enough of the client side - now for the server! Most of the world uses Apache and I do too (except for Lighttpd on my hacked MyBook World server).

Besides the HTTP verbs, the other major component to REST is resources. We have already been using them as "http:people/jan", "http:people/fred" where the URL labels the resources "Jan" and "Fred". REST assumes that URLs are identifying resources, which may be documents, people, items on a shopping cart, etc.

So why didn't we use URLs such "http:people/get.php?jan" when we want to do a GET? Simple: REST URLs label resources not actions on these resources. get.php is an action on some parameter rather than a label. The value of using resources for URLs is that we can perform different actions on the same resource as in "GET people/jan", "DELETE people/jan" etc, and each time we refer to the same resource. This would not be clear if we had "GET people/get.php?jan" and "PUT people/put.php?name=jan&id=1".

In addition, just for maintenance reasons, we would want to separate implementation from data: suppose we decided to move away from PHP and use Ruby on Rails. We wouldn't want to revise all of our implementation-specific URLs - that should be an action behind the scenes in how to handle our resource URLs.

That said, how do we go about mapping resource URLs into actions on those resources? There are several methods. I've chosen Apache's Rewrite rules. These are specified in Apache configuration files, such as /etc/apache2/sites-enabled/000-default on an Ubuntu system. A rule consists of two components: a condition to be satisfied and then a rewrite of the URL. The rewrite rule uses regular expressions as in Perl, where "^(.*)$" means any characters between a begin and end of a string.

I have rules that apply only to the directory on my server box where I keep the "people" files. The Apache directive is

        <Directory /home/httpd/html/boxhill/ict329/webservices/people/>
                RewriteEngine on

                RewriteCond %{REQUEST_METHOD} =GET
                RewriteRule ^(.*)$ get_request.php?$1 [QSA]

                RewriteCond %{REQUEST_METHOD} =PUT
                RewriteRule ^(.*)$ put_request.php?$1 [QSA]

                RewriteCond %{REQUEST_METHOD} =DELETE
                RewriteRule ^(.*)$ delete_request.php?$1 [QSA]
        </Directory>

which calls get/put/delete_request.php with the person's name extracted from the URL and appended as a parameter to the PHP call. After you have made changes like this to the Apache configuration files don't forget to restart or refresh Apache (kill -HUP `cat /var/run/apache2.pid`).

My database

For this blog, I use a MySQL table with two fields, name (text) and id (integer).

PHP

The PHP code is fairly straightforward. It just extracts the parameter information from the $_REQUEST, accesses the database and returns the results using the appropriate HTTP return codes.

The get.php program is

<?php
$user="*****";
$password="*****";
$database="*****";
$localhost="localhost";
mysql_connect($localhost,$user,$password);
@mysql_select_db($database) or die( "Unable to select database");

$req=$_REQUEST;
$keys=array_keys($req);
$name=mysql_real_escape_string($keys[1]);
if ($name=="") {
   $query="SELECT * FROM people";
} else {
  $query="SELECT * FROM people WHERE name=\"$name\"";
}

$result=mysql_query($query);
mysql_close();

$num=mysql_num_rows($result);
if ($num == 0) {
   header("HTTP/1.1 404 Not Found");
   exit();
}

$n=0;
while ($n < $num) {
      $id=mysql_result($result,$n,"id");
      $name=mysql_result($result,$n,"name");

      echo "$name\n";    
      echo "$id\n";

      $n++;
}
?>
The delete_request.php is slightly more complex as we have to determine if the DELETE  has succeeded by counting if the number of rows affected is zero or not.

<?php
$user="*****";
$password="******";
$database="******";
$localhost="localhost";
mysql_connect($localhost,$user,$password);
@mysql_select_db($database) or die( "Unable to select database");

$req=$_REQUEST;
#print_r($req);
$keys=array_keys($req);
$name=mysql_real_escape_string($keys[1]);

$query="DELETE FROM people WHERE name=\"$name\"";

$result=mysql_query($query);
$num=mysql_affected_rows();
mysql_close();

if ($num == 0) {
   header("HTTP/1.1 404 Not Found");
} else {
  header("HTTP/1.1 202 OK");
}
?>

The put_request.php is also complicated by needing to tell if we are creating (INSERT) or modifying a row (exists, DELETE, INSERT). It is


<?php
$user="*****";
$password="******";
$database="*****";
$localhost="localhost";
mysql_connect($localhost,$user,$password);
@mysql_select_db($database) or die( "Unable to select database");

$req=$_REQUEST;
$keys=array_keys($req);
$name=mysql_real_escape_string($keys[1]);

$putdata = fopen("php://input", "r");
$ID = intval(fread($putdata, 1024));
fclose($putdata);

# are we updating?
$query="SELECT * FROM people WHERE name=\"$name\"";    
$result=mysql_query($query);

$num=mysql_num_rows($result);
if ($num == 0) {
   header("HTTP/1.1 201 Created"); # creating
} else {
   header("HTTP/1.1 204 OK"); # updating
   # delete current row first
   $query="DELETE FROM people WHERE name=\"$name\"";

   $result=mysql_query($query);
}

# now add new person
$query="INSERT people VALUES (\"$name\", $ID)";

$result=mysql_query($query);
mysql_close();

?>

Summary

This blog has discussed beginning to end of a web application using REST. Hope it is useful!

If you like this blog, please contribute using Flattr


or donate using PayPal





Wednesday, 4 May 2011

Ubuntu 11.04

I've got a laptop running Ubuntu 10.10, a netbook running 10.04 and as an alternative boot, the 10.10 netbook remix and also a rather odd system: I'm using an HP thin client as media centre in my lounge connected to a 58 inch Panasonic TV via DVI to HDMI. It streams from a MyBook World NAS that I hacked into to become a general server.

The TV screen is about 3 metres away from where I sit, so I want a "10 foot GUI". Surprisingly perhaps some of the netbook distros do very well at that as while they are designed for small screens they also do well for big screens a long way away. Currently I am using xPud which I have customised to my environment.

But I have to keep rebuilding my version of xPud to keep up, so I am always trying other distros with an easier management cycle. So of course Ubuntu 11.04 looks like a nice candidate.

It's worse than a disappointment, it doesn't display a usable screen at all. Bits of icons smeared across the screen and the left tool bar only showing when you click on the right-hand-side (!) of the screen.

It runs okay on my netbook, but I'm loathe to upgrade my main laptop while there are such problems with other systems. I guess I will wait till 11.10 for Unity to stabilise before doing a general upgrade to Ubuntu 11.

Sunday, 3 April 2011

Shell scripts to handle filenames with spaces

Posix/Unix/Linux was not designed to handle filenames with spaces in them. However, Linux and Windows filesystems allow them and also many other "funny" characters. This has been brewing as a topic in Linux Journal recently, and Dave Taylor has just written an article on it in the February, 2011 issue. He spots files with spaces in them by the shell pattern "*\ *" and then mucks around changing spaces into other things. It's good stuff, but overkill for some cases.


For a long time now, I've been writing scripts that handle filenames both with and without spaces. You've got to know your shell and how Posix commands work! Commonly, I want to list files in a directory and do things to them whether or not they have spaces in them.

Shell patterns such as "*" break strings into "words" based on whitespace (spaces, tabs, newlines). This stuffs up a filename if its has spaces in it, since the name then gets split into separate words. But commands such as "ls" (when not directed to a terminal) list each filename on a separate line. So if you have something that distinguishes between spaces/tabs and newlines then you can get complete filenames with or without spaces.

The shell command "read" reads a line and breaks it into words. so
 read a b c
with input
 a line of text
will assign
 a="a"
 b="line"
 c="of text"
Just
  read line
will read all of the line into the variable. It stops reading on end-of-line so it has the distinction type I often need.

But how to use it? Well, the shell while loop is just a simple command, and as such can have its I/O redirected.  So I do this:
 ls |
 while read filename
 do
    #process filename e.g.
    cp "$filename" ~/backups
 done
This works for all files, with or without spaces. Just don't forget the quotes while processing the file! 

Of course, this doesn't work for all uses: note the find and xargs combination that Dave also commented on:
 find . -print0 | xargs -0 ...

Saturday, 2 April 2011

HTML 5 has a serious flaw

HTML 5 is long overdue, after the WWW Consortium's failed attempt at convincing us to use XHTML. It has many useful features, but one glaring fault: it has discarded version control. I've been writing and designing distributed systems for over twenty years, and one thing has become very clear: if you don't include version numbers in your protocol then you are asking for trouble.

The document type has been simplified. Before it used to have horrible things like
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" 
"http://www.w3.org/TR/html4/loose.dtd">

But now it has been simplified overmuch to just

<!DOCTYPE html>
There are some people who like this (e.g. John Resig). But I don't. HTML will continue to evolve - there will be new tags and attributes, and the existing behaviour will be clarified or changed. But without a version number, how will a browser (or any user agent) be able to work out which version it is dealing with? And how can a content generator signal which version it is creating? Already there is considerable confusion about which bits of HTML 5 are supported by different browsers.

The simple answer is perhaps that this allows vendors free reign to do what they want - and we saw what a mess that caused before HTML 4 put a standard in the ground. There is still time for the WWW Consortium to fix at least this one error before it is too late.