Subject: On user feedback

From: shd
Date: Mon Jan 2 18:38:59 2006

To: Kohina-UADE-Forum
First, it seems there is some misconception among users that I can know that some compiler errors and warnings exist. I do not! uade compiles cleanly without warnings on my system, so if you ever get errors and warnings, please send me email or post on this forum.

Second, the thumb rule is that only problems that you report to me are ever fixed. If you do not report problems then do not expect them to be fixed. If you do report problems, then I will at least make them public (by adding to doc/BUGS) so that someone else may fix the problem if I can't or wont.

Bottom line. If you care about anything at all in uade, then please report problems if they arise. I think open source in general can only be as good as the amount of problem feedback it gets.

&lt;damnit&gt;I won't fix any more bugs unless reported ;-)&lt;/damnit&gt;

---

From: amadeus
Date: Sat Jan 7 07:21:41 2006

To: Kohina-UADE-Forum
[quote:f7dc3c8618="shd"]First, it seems there is some misconception among users that I can know that some compiler errors and warnings exist. I do not! uade compiles cleanly without warnings on my system, so if you ever get errors and warnings, please send me email or post on this forum.
[/quote:f7dc3c8618]

I doubt many programs can say the same =)

CVS compiles here with no errors on Fedora Core 4.

But I still get the
[code:1:f7dc3c8618]ALSA&#58; underrun, at least 0ms.n subsong 0[/code:1:f7dc3c8618]
with uade123 on every song.

---

From: shd
Date: Sat Jan 7 14:16:09 2006

To: Kohina-UADE-Forum
[quote:84fbca594f="amadeus"]
But I still get the
[code:1:84fbca594f]ALSA&#58; underrun, at least 0ms.n subsong 0[/code:1:84fbca594f]
with uade123 on every song.[/quote:84fbca594f]

It could be that libao and ALSA are sensitive to loads on some systems. What are specs of your machine? processor, sound card and kernel?

Does it happen many times during a song or just once? And does it happen in the beginning or something? If you issue two songs does the buffer overrun happen between the songs?

---

From: amadeus
Date: Sat Jan 7 17:51:04 2006

To: Kohina-UADE-Forum
[quote:7850110372="shd"]It could be that libao and ALSA are sensitive to loads on some systems. What are specs of your machine? processor, sound card and kernel?[/quote:7850110372]

[code:1:7850110372]Linux 2.6.14-1.1653_FC4 #1 Tue Dec 13 21&#58;32&#58;09 EST 2005 i686 athlon i386 GNU/Linux[/code:1:7850110372]

My soundcard is a Creative Labs PCI 128 and regonized as:
[code:1:7850110372]Vendor&#58; Ensoniq
Model&#58; 5880 AudioPCI
Modele&#58; snd-ens1371
[/code:1:7850110372]

[quote:7850110372="shd"]Does it happen many times during a song or just once? And does it happen in the beginning or something? If you issue two songs does the buffer overrun happen between the songs?[/quote:7850110372]

I get about 10 of those lines evertime I play a new format, and whenever I change workspace or e.g. change tabs in Firefox.

It somehow seams as if, it only happens when I access something in RAM that haven't been used for awhile.

When it happens the sound is out of sync while the lines are written. Just like if you play a wave file from harddisk, and do some heavy disk activity at the same time.

---

From: Guest
Date: Sat Jan 7 18:47:34 2006

To: Kohina-UADE-Forum
Here is a Flash movie of the bug, where I scroll in Firefox.
[url]http://www.heko.dk/~maj/out.swf[/url]

I have used vnc2swf, and I an quite sure it can capture the audio aswell, but I can't find any documentation on how to do that =(

---

From: shd
Date: Sat Jan 7 20:53:30 2006

To: Kohina-UADE-Forum
[quote:99a90c945c="amadeus"]
I get about 10 of those lines evertime I play a new format, and whenever I change workspace or e.g. change tabs in Firefox.

When it happens the sound is out of sync while the lines are written. Just like if you play a wave file from harddisk, and do some heavy disk activity at the same time.[/quote:99a90c945c]

Do you have memory pressure which causes swapping? Note that executables and libraries are implemented as separate swaps on Linux. Each exe and library file is accessed through a memory-map which acts like a swap (but nothing is of course written there).

If no memory pressure, then have you asked libao developers about its buffering issues?

---

From: amadeus
Date: Sat Jan 7 21:12:48 2006

To: Kohina-UADE-Forum
[quote:f0fd20c548="shd"]Do you have memory pressure which causes swapping? Note that executables and libraries are implemented as separate swaps on Linux. Each exe and library file is accessed through a memory-map which acts like a swap (but nothing is of course written there).

If no memory pressure, then have you asked libao developers about its buffering issues?[/quote:f0fd20c548]

I have 1GB of memory, and plenty of free. I haven't seen my swap been used yet. This is top when running uade123

[code:1:f0fd20c548]top - 22&#58;01&#58;47 up 15&#58;53,  7 users,  load average&#58; 0.30, 0.32, 0.38
Tasks&#58;  99 total,   2 running,  97 sleeping,   0 stopped,   0 zombie
Cpu&#40;s&#41;&#58; 15.3% us,  1.3% sy,  0.0% ni, 83.1% id,  0.0% wa,  0.3% hi,  0.0% si
Mem&#58;    905832k total,   864828k used,    41004k free,    56848k buffers
Swap&#58;   250448k total,       60k used,   250388k free,   479932k cached
[/code:1:f0fd20c548]

There are very few reported bugs in libao. Should I file one?
[url]http://trac.xiph.org/report/13[/url]

Btw. When the bug is active 100% CPU is used for 1-2 sec.

---

From: shd
Date: Sat Jan 7 22:40:10 2006

To: Kohina-UADE-Forum
[quote:5332601059="amadeus"]
Btw. When the bug is active 100% CPU is used for 1-2 sec.[/quote:5332601059]

I notice that the same problem occurs if I open many tabs into firefox and load many pages there and then press ctrl-page up so that firefox will rapidly change between pages. The music practically halts. Crap! This needs to be investigated..

---

From: amadeus
Date: Sat Jan 7 23:30:02 2006

To: Kohina-UADE-Forum
[quote:dabfe1781c="shd"][quote:dabfe1781c="amadeus"]
Btw. When the bug is active 100% CPU is used for 1-2 sec.[/quote:dabfe1781c]

I notice that the same problem occurs if I open many tabs into firefox and load many pages there and then press ctrl-page up so that firefox will rapidly change between pages. The music practically halts. Crap! This needs to be investigated..[/quote:dabfe1781c]

Does that take 100% on your computer, when uade123 is not running?

I don't have to make my computer max out. If uade123 isn't running the proceedure I did in the Flash movie doesn't take 100% cpu load.

I can reproduce it everytime:

* Load 5 websites in tabs
* Change workspace.
* Start uade123
* Change to Firefox workspace.
* Left click on the tabs.

Can physical memory be fracmented like swap partitions?

---

From: shd
Date: Sat Jan 7 23:38:35 2006

To: Kohina-UADE-Forum
[quote:584d3cb154="amadeus"]
Does that take 100% on your computer, when uade123 is not running?
[/quote:584d3cb154]

Only when I'm rapidly switching tabs. It's almost 100%. But I think that so it should be, but I can't see any reason why uade123 and uadecore wouldn't get a fair time slice (they only require 5% of total CPU or so).

[quote:584d3cb154]Can physical memory be fracmented like swap partitions?[/quote:584d3cb154]

From kernel point of view physical memory is continuous, except perhaps on some rare platforms. From process point of view (logical memory) it's only continuous inside one page (4 KiB usually). This bug is definitely not about virtual memory.

---

From: amadeus
Date: Sat Jan 7 23:47:14 2006

To: Kohina-UADE-Forum
[quote:60b9fc3fc2="shd"]
[/quote:60b9fc3fc2]

Could it be, that we are seeing two different bugs here? I don't have to max out my CPU inorder to tricker the bug, which you have to?

On my computer it is something inside UADE or something UADE uses that maxes my CPU out.

[quote:60b9fc3fc2]
From kernel point of view physical memory is continuous, except perhaps on some rare platforms. From process point of view (logical memory) it's only continuous inside one page (4 KiB usually). This bug is definitely not about virtual memory.[/quote:60b9fc3fc2]

Okay, so is that out of the question.

When I think about it, I think I first saw it in the 1.5 series. Do you still have those archives for download? I won't mind compiling them and test.

I have checked UADE website, and there are only 1.0x and 2.0x archives.

---

From: shd
Date: Sat Jan 7 23:56:22 2006

To: Kohina-UADE-Forum
[quote:40e556c5bb="amadeus"]
When I think about it, I think I first saw it in the 1.5 series. Do you still have those archives for download? I won't mind compiling them and test.[/quote:40e556c5bb]

This doesn't happen with ogg123 which uses libao. I suspect uadecore gets scheduled too infrequently by the kernel. libao ALSA buffers drain out before uadecore has time to synthesize more sample data, or that's what I believe happens. This just became my highest priority problem with UADE.

[quote:40e556c5bb]
I have checked UADE website, and there are only 1.0x and 2.0x archives.[/quote:40e556c5bb]

I would bet all 1.5* versions share the same problem. http://zakalwe.virtuaalipalvelin.net/uade/uade2/devel/

---

From: shd
Date: Sat Jan 7 23:59:44 2006

To: Kohina-UADE-Forum
[quote:24d6b4230c="amadeus"]
Could it be, that we are seeing two different bugs here? 
[/quote:24d6b4230c]

I guess not. jraitala complained about underruns with Mac OS X, and I would guess this is the same problem. This should affect all platforms if my guess is correct.

---

From: amadeus
Date: Sun Jan 8 00:49:26 2006

To: Kohina-UADE-Forum
[quote:c6e2502efb="shd"]I would bet all 1.5* versions share the same problem. http://zakalwe.virtuaalipalvelin.net/uade/uade2/devel/[/quote:c6e2502efb]

You are right. 1.5pre1 have this bug aswell.

---

From: Vlcice
Date: Sun Jan 8 02:59:01 2006

To: Kohina-UADE-Forum
[quote:4063637bbe="shd"][quote:4063637bbe="amadeus"]
Could it be, that we are seeing two different bugs here? 
[/quote:4063637bbe]

I guess not. jraitala complained about underruns with Mac OS X, and I would guess this is the same problem. This should affect all platforms if my guess is correct.[/quote:4063637bbe]

Yes, it seems to be a general libao issue, at least as far as I can tell. In the older commandline UADE versions, I had audio skipping while scrolling and so forth when using libao, but not when using the CoreAudio driver.

---

From: shd
Date: Sun Jan 8 03:09:28 2006

To: Kohina-UADE-Forum
[quote:564ef07b8f="amadeus"]
You are right. 1.5pre1 have this bug aswell.[/quote:564ef07b8f]

I think I've located the problem. We should issue a new sample data request for the emulator before passing current sample data to libao. This way both libao and the emulator can do useful work simultaneously. If that approach doesn't work then I'm very much at loss because ogg123 works well with libao. With our current approach, increasing buffering in the emulator would actually only worsen the problem and that was of course the first fix I tried =) So there's a definite systematic problem in the playloop I wrote.

---

From: shd
Date: Sun Jan 8 14:00:27 2006

To: Kohina-UADE-Forum
[quote:92a92ebde6="shd"]
I think I've located the problem. We should issue a new sample data request for the emulator before passing current sample data to libao. This way both libao and the emulator can do useful work simultaneously. If that approach doesn't work then I'm very much at loss because ogg123 works well with libao. With our current approach, increasing buffering in the emulator would actually only worsen the problem and that was of course the first fix I tried =) So there's a definite systematic problem in the playloop I wrote.[/quote:92a92ebde6]

Unfortunately it seems this is not the whole problem. It is definitely a real problem that needed to be fixed, and it is fixed in the current CVS version. The fix shows there is actual improvement, but I still get underruns. I added a remark to ChangeLog:

[code:1:92a92ebde6]
This didn't solve the underrun problem completely but effect of improving behavior is clearly observable with 'top'. Before this change, top showed only 0.4% CPU usage for uadecore, but after the change it shows 3.3% CPU usage, and the latter reflects reality much better.
[/code:1:92a92ebde6]

So now uade123 and the uadecore behave sanely with respect to operating system kernel. The next thing is to figure how to set buffers properly or do something else to fix this dawned thing.

---

From: shd
Date: Sun Jan 8 16:43:33 2006

To: Kohina-UADE-Forum
[quote:32d34db311="amadeus"][quote:32d34db311="shd"]I would bet all 1.5* versions share the same problem. http://zakalwe.virtuaalipalvelin.net/uade/uade2/devel/[/quote:32d34db311]

You are right. 1.5pre1 have this bug aswell.[/quote:32d34db311]

Although my newest fix wasn't complete, running uade as root nice -19 one doesn't get any underruns =) Not exactly multitasking friendly priority..

---

From: shd
Date: Sun Jan 8 17:01:41 2006

To: Kohina-UADE-Forum
[quote:707ee1eb84="amadeus"][quote:707ee1eb84="shd"]I would bet all 1.5* versions share the same problem. http://zakalwe.virtuaalipalvelin.net/uade/uade2/devel/[/quote:707ee1eb84]

You are right. 1.5pre1 have this bug aswell.[/quote:707ee1eb84]

Would you try the latest CVS version and tell if it produces more, same or less amount of underruns. The worst case hasn't improved here, but it's perhaps possible that normal operations do not cause underruns.

---

From: amadeus
Date: Sun Jan 8 17:48:31 2006

To: Kohina-UADE-Forum
[quote:12f2bfd265="shd"]Would you try the latest CVS version and tell if it produces more, same or less amount of underruns. The worst case hasn't improved here, but it's perhaps possible that normal operations do not cause underruns.[/quote:12f2bfd265]

It is still there, when I scroll, switch workspace, press links in Firefox, but the time the module is out-of-sync is much shorter now, so it doesn't quite reach 100% CPU load on my AMD XP 1,9Ghz.

---

From: amadeus
Date: Sun Jan 8 17:53:03 2006

To: Kohina-UADE-Forum
[quote:e373a695f9="shd"]Although my newest fix wasn't complete, running uade as root nice -19 one doesn't get any underruns =) Not exactly multitasking friendly priority..[/quote:e373a695f9]

With the latest CVS running uade123 is root, the bug is still there, and the number of underrun lines is almost twice as many compared to my regular user.

I haven't tried the old CVS as root, so perhaps this is the same there?

---

From: shd
Date: Sun Jan 8 18:00:13 2006

To: Kohina-UADE-Forum
[quote:229e586893="amadeus"][quote:229e586893="shd"]Would you try the latest CVS version and tell if it produces more, same or less amount of underruns. The worst case hasn't improved here, but it's perhaps possible that normal operations do not cause underruns.[/quote:229e586893]

It is still there, when I scroll, switch workspace, press links in Firefox, but the time the module is out-of-sync is much shorter now, so it doesn't quite reach 100% CPU load on my AMD XP 1,9Ghz.[/quote:229e586893]

Oh well.. I suppose I have to rethink the token-passing messaging if I don't figure out any flaw here. I tried to create a logically simple IPC protocol and the reality struck me down :(

---

From: amadeus
Date: Sun Jan 8 18:25:58 2006

To: Kohina-UADE-Forum
[quote:f7c013bc16="shd"]Oh well.. I suppose I have to rethink the token-passing messaging if I don't figure out any flaw here. I tried to create a logically simple IPC protocol and the reality struck me down :([/quote:f7c013bc16]

Sounds like a major code rework? =(

Would it be possible to get ideas from GPL software, or is score way too different in its arch.?

---

From: shd
Date: Mon Jan 9 02:56:58 2006

To: Kohina-UADE-Forum
[quote:4e8a9f6666="amadeus"]
Sounds like a major code rework? =(
[/quote:4e8a9f6666]

No. But since I've already tried two times to fix the problem without success, I'm not sure what this will lead to exactly. Hopefully I will learn something.

The frontend has been "pulling" data from uadecore, which causes some extra latency and short idle moments in the uadecore, but I'll make it a "push" architecture so that uadecore "pushes" data to the frontend all the time. Pushing stops when frontend says "hold on" or something interesting happens. This would be like unix command line pipes, but there is rare messages to the opposite direction as well (subsong change or song end command).

---

From: amadeus
Date: Mon Jan 9 05:33:16 2006

To: Kohina-UADE-Forum
[quote:bfe1e15e49="shd"]The frontend has been "pulling" data from uadecore, which causes some extra latency and short idle moments in the uadecore, but I'll make it a "push" architecture so that uadecore "pushes" data to the frontend all the time. Pushing stops when frontend says "hold on" or something interesting happens. This would be like unix command line pipes, but there is rare messages to the opposite direction as well (subsong change or song end command).[/quote:bfe1e15e49]

If the pushing approach works, will the worst case senario when be a slight delay on commands like subsong change?

Although it is not clear to me, why the current approach fails? I mean; When the desktop is left alone, everything works out fine, and low CPU usage.

Why does uade123 then crumble when other programs are used? Wouldn't a 1.9GHz CPU (in my case :) ) give plenty of cycles to uade123 on a Tonken Ring communication?

---

From: shd
Date: Thu Jan 12 20:16:23 2006

To: Kohina-UADE-Forum
I put a bug report to: http://bugme.osdl.org/show_bug.cgi?id=5877

If you have some enlightening experiences and data with respect to that bug, please add comments there. I used 'strace -t' to analyze the problem. There's a full wall clock second without any syscalls happening in uade, so I suspect it's a scheduler starvation problem.

Do you have 2.4.x by any chance? It would be interesting to test it.

---

From: shd
Date: Fri Jan 13 01:09:13 2006

To: Kohina-UADE-Forum
[quote:034e2b59ad="shd"]I put a bug report to: http://bugme.osdl.org/show_bug.cgi?id=5877

Do you have 2.4.x by any chance? It would be interesting to test it.[/quote:034e2b59ad]

My bug report paid off. It doesn't happen on 2.6.12 so now it is known that it's some kind of kernel scheduling problem.