Subject: uade123: How does recognition by content get triggered?

From: Guest
Date: Thu May 18 10:02:40 2006

To: UADE-Developer-discussion
Some of the tunes in the ExoticA collection don't get recognised by their prefix or content, and are rejected out of hand. So I investigated with uade123 2.02 to see what happens when I remove all prefixes from the config file.

The executive message of this posting is &quot;please can I have a command-line option to force recognition of files by going through all the deliplayer check functions, when uade's internal filename/contents recogntion fails.&quot;

First, I removed all &quot;prefixes=&quot; lines in /usr/local/share/uade2/eagleplayer.conf. I always get this error: &quot;uade123: ../common/eagleplayer.c:596: uade_read_eagleplayer_conf: Assertion `exti &lt; ps-&gt;nextensions' failed. &quot; - looks like a corner case in the code. It also always fails if the very last entry in eagleplayer.conf does not have prefixes set, but always succeeds as long as the last entry does have prefixes set. So I gave the last entry a &quot;prefixes=blahblahblah&quot; line.

I then ran uade123 in verbose mode and noticed that the additional recognition (file magic) is getting called, e.g. &quot;FC13.Thalion: deduced extension: FC13&quot;, but then that can't be mapped to the player: &quot;Deduced file extension (FC13) is not on the uadeformats list.&quot; - I think that's a good idea, it's a useful feature to be able to configure which player even the results of recognition should point to, instead of hard-coding it into the executable. I also can't think of a real reason why someone would want to take these deduced extensions out of the config file. However, as a last-ditch attempt to play an otherwise unrecognised file, I would greatly appreciate if there was an option to go through all the deliplayers'/eagleplayers' DT_CHECK and DT_CHECK2 functions, as the real DeliTracker or EaglePlayer would.

I also had some trouble trying to automate the process of testing:
[code:1:433276d24a]$ find . -type f -print0 | xargs -0 uade123 --verbose &gt;../uade-log.txt
uade123&#58; can't setup interactive mode&#58; Inappropriate ioctl for device
xargs&#58; uade123&#58; exited with status 255; aborting[/code:1:433276d24a]
I replaced the &quot;exit()&quot;s with simple returns in terminal.c. However, in that case, uade doesn't respond to the usual UNIX signals, e.g. &quot;uade --verbose */* | head&quot; does not quit uade after 10 lines of output.

Without file extension recognition, uade123 didn't recognise 2189 Amiga music files (not the oggs and mp3s), which cover roughly 79 file formats. The results are here: [url]http://kyz.mine.nu/misc/unexotica_tunes_unrecognised_by_uade123.txt[/url]

---

From: Guest
Date: Thu May 18 23:06:12 2006

hi kyz,

thanks for the list. Yeah, there's a lot missing in amifilemagic. It started out as a way sort out TFMX files and became a nightmare to maintain ;-)

I take out following files being instruments/sample/replaycode files.
[code:1:4b41aba6b0]Dizzy_Panic/mod.daisy.nt
Dizzy_Panic/mod.fx-chip.nt
Dizzy_Panic/mod.gameover.nt
Dizzy_Panic/mod.secret.nt
Dizzy_Panic/mod.title.nt

Fiendish_Freddys/songplay

Mandelbrot_Show/Instruments/Never1.Instr
Mandelbrot_Show/Instruments/Never2.Instr
Mandelbrot_Show/Instruments/Square.Instr

Overlander/mcs.Overlander

Steg_the_Slug/mod.fx-chip.nt
Steg_the_Slug/mod.gameover.nt
Steg_the_Slug/mod.hiscore.nt
Steg_the_Slug/mod.intro.nt
Steg_the_Slug/mod.level1.nt
Steg_the_Slug/mod.level2.nt
Steg_the_Slug/mod.level3.nt
Steg_the_Slug/mod.secret.nt
Steg_the_Slug/mod.title.nt
Steg_the_Slug/mod.welldone.nt

Wings/Instruments/Acord1
Wings/Instruments/BsDrum
Wings/Instruments/Clarnt
Wings/Instruments/Cymbal
Wings/Instruments/FHorn2
Wings/Instruments/Horns
Wings/Instruments/Horns1
Wings/Instruments/Piano2
Wings/Instruments/Piano3
Wings/Instruments/Piano4
Wings/Instruments/Picolo
Wings/Instruments/PSnare
Wings/Instruments/PTrump
Wings/Instruments/Square
Wings/Instruments/Strng2
Wings/Instruments/Trmbon
Wings/Instruments/Tuba
Wings/Instruments/Tympny[/code:1:4b41aba6b0]

Of the rest the mods puzzle me most. They should have worked just until last week, until me and Heikki removed any Filelogic from the protracker player to configure the replayer from the eagleplayer.conf file.

About a possibility to have the old delitracker behaviour to rely on the replayers checks... Not sure. I think it could be nice to have but it will slow down things, especially when adding tunes to xmms or audacious even more I guess.

Michael

---

From: Guest
Date: Fri May 19 17:10:05 2006

To: UADE-Developer-discussion
[quote=&quot;Anonymous&quot;]hi kyz,

[/code]

Ah, I knew I would miss some! I already spent quite some time pulling out those things :)

[quote:acf5d49629]
[/quote:acf5d49629]

This is uade123 2.02, in case I didn't mention.

[quote:acf5d49629]

Michael[/quote:acf5d49629]

It would certainly slow down things, which is why I think it should only be a configurable option and/or only used after regular file recognition has failed and uade would otherwise reject the file. But it should certainly exist, otherwise why do player authors write check routines?

---

From: shd
Date: Sat May 20 09:27:53 2006

To: UADE-Developer-discussion
[quote:37caea42de="Anonymous"]The executive message of this posting is &quot;please can I have a command-line option to force recognition of files by going through all the deliplayer check functions, when uade's internal filename/contents recogntion fails.&quot;
[/quote:37caea42de]

It's better to do file detection in native code, otherwise file checks will cost tens of millions of cycles. Fixing detection problems in native code is our priority. Detecting by using eagleplayers check() method does not fit current design, but it would be doable though. Best way to do would be launching a separate emulator running only detection code from eagleplayers :-) Also, it would take many megabytes more memory. Currently playing with uade123 takes only ~23 MiB of virtual space (much less RSS) on amd64.

[quote:37caea42de]First, I removed all &quot;prefixes=&quot; lines in /usr/local/share/uade2/eagleplayer.conf. I always get this error: &quot;uade123: ../common/eagleplayer.c:596: uade_read_eagleplayer_conf: Assertion `exti &lt; ps-&gt;nextensions' failed. &quot; - looks like a corner case in the code.
[/quote:37caea42de]

I'll try to fix this.

[quote:37caea42de]I then ran uade123 in verbose mode and noticed that the additional recognition (file magic) is getting called, e.g. &quot;FC13.Thalion: deduced extension: FC13&quot;, but then that can't be mapped to the player: &quot;Deduced file extension (FC13) is not on the uadeformats list.&quot;
[/quote:37caea42de]

The logic in file detection is not straightforward. Prefixes in eagleplayer.conf are actually needed whether or not file is detected by prefix, because the prefixes in eagleplayer.conf define internal classification strings for different formats.

The content detection routine may return &quot;fc14&quot; and then fc14 prefix is searched and it is determined to be FutureComposer1.4. However, if the content detection routine is unable to decide the format, it returns an empty string. The empty string is handled in one of two ways:
1. Take the filename prefix from the real file and search it from the eagleplayer.conf. Use the associated player if &quot;content_detection&quot; is not specified for it.
2. Reject the format because prefix detection is not allowed at all. See &quot;magic_detection&quot; in uade.conf or use -m

So the misunderstanding here is that prefixes are two things: internal classifiers and name prefixes.

If you really want to force player for one specific song, use -P or use song.conf:
[code:1:37caea42de]uade123 --set=player=MyPlayer foo.bar
[/code:1:37caea42de]

will teach uade123 to use MyPlayer in the future, but it must be copied to the eagleplayer path.

[quote:37caea42de]
I also had some trouble trying to automate the process of testing:
[code]$ find . -type f -print0 | xargs -0 uade123 --verbose &gt;../uade-log.txt
[/quote:37caea42de]

Fixed the CVS version to use /dev/tty instead of stdin and not to be confused if terminal setup fails.

You are aware of -r option I assume?

[quote:37caea42de] However, in that case, uade doesn't respond to the usual UNIX signals, e.g. &quot;uade --verbose */* | head&quot; does not quit uade after 10 lines of output.
[/quote:37caea42de]

It does die after head dies, but you don't get output (or line feeds) to stdout, which is the problem. Try uade123 -v * 2&gt;&amp;1 |head and it works as expected.

---

From: shd
Date: Sat May 20 10:39:06 2006

To: UADE-Developer-discussion
[quote:87f62a3077="Anonymous"]
First, I removed all &quot;prefixes=&quot; lines in /usr/local/share/uade2/eagleplayer.conf. I always get this error: &quot;uade123: ../common/eagleplayer.c:596: uade_read_eagleplayer_conf: Assertion `exti &lt; ps-&gt;nextensions' failed. &quot;
[/quote:87f62a3077]

Now it doesn't exit with failure, but gives an ugly warning. One should not remove prefixes but possibly add &quot;content_detection&quot; or comment the line.