Discogs: Parse Album (Release) Options

[DetlevD]

... First priority is to be able to convert Discogs XML file with less hustle for user. ...

Could you please elaborate this idea in more detail?

DD.20110412.1613.CEST

[Victor Kostas]

Could you please elaborate this idea in more detail?

DD.20110412.1613.CEST

Currently, implemented Web source scripts are split in 2 steps:
1. Parse candidate releases (ParseIndex). Discogs provides HTML results but no XML ones.
2. Parse selected release (ParseAlbum). Discogs provides both HTML and XML results.

Thanks to your suggestion (use Local Web Server) I came with one idea: AlbumURL (Parse no 2) to not request Release directly from Discogs but from local Web Server instead. Local server will request the release info (XML format) from Discogs. The XML will be amended and then returned to mp3tag Web Source. A PHP script must be implemented in the local Web Server.
Nothing is changed in the way Web source works.

There are plenty Local Web Servers in the net. I prefer portable versions (no installation required).
I used XAMPP package which contains Web Server, PHP, FileZilla and others.

What do you think?

[DetlevD]

... I came with one idea: AlbumURL (Parse no 2) to not request Release directly from Discogs but from local Web Server instead. Local server will request the release info (XML format) from Discogs. The XML will be amended and then returned to mp3tag Web Source. A PHP script must be implemented in the local Web Server.
Nothing is changed in the way Web source works.
There are plenty Local Web Servers in the net. I prefer portable versions (no installation required).
I used XAMPP package which contains Web Server, PHP, FileZilla and others.
What do you think?

I think ... well ... today many people are permanently linked together via several networks and services ... and they all have open doors to their harddisks ... but you cannot demand it from a normal user that he agrees to the installation of a web server on his home computer.
There might be a simpler way.
No extra web server, just a simple HTTP requester.

Maybe of interest ... there is a source where different access methods and tools are described ...
http://techsupt.winbatch.com/webcgi/webbat...h~Web~Pages.txt
Could be XMLStarlet of some use?

DD.20110412.1748.CEST

This post has been edited by DetlevD: Apr 12 2011, 16:48

[Victor Kostas]

I think ... well ... today many people are permanently linked together via several networks and services ... and they all have open doors to their harddisks ... but you cannot demand it from a normal user that he agrees to the installation of a web server on his home computer.
There might be a simpler way.
No extra web server, just a simple HTTP requester.

Maybe of interest ... there is a source where different access methods and tools are described ...
http://techsupt.winbatch.com/webcgi/webbat...h~Web~Pages.txt
Could be XMLStarlet of some use?

DD.20110412.1748.CEST

I will have a look. Thanks.
In general my goal is to implement an intermediate script between (discogs) XML and mp3tag Web source script Album parser. Thus, album XML parser script will be simplified too.
Of course installation of local web server is a disadvantage.
Thank you again for your findings.

[Victor Kostas]

I have finished (a beta version) of 2 PHP scripts which will help a lot with discogs site XML data.
1st script is used to filter releases of one given artist. Currently discogs returns all artist releases in a big XML without giving the option of any kind of filter... Well until now.

2nd script is "normalization" of discogs release XML.
Distribute Extraartists in track level.
Calculate artistmulti for each artists tag.
Calculate composer, lyricist, involvedartistst in track level.

With those 2 scripts we can deal only with (amended) XML files. Scripts can be easily amended to keep mp3tag webscripts simple.

With a local PHP server it is pretty easy.
I will come with more details.
Any thoughts?

[pone]

So your script can only be used if you have a local php server? Do you need to download the discogs data first?

Have you seen that discogs has a new API (v2.0) and the old one will be shut down in December?
http://forums.mp3tag.de/index.php?showtopic=14385
http://blog.discogs.com/2011/06/api-v20.html

I hope that doesn't affect you script.

[Victor Kostas]

So your script can only be used if you have a local php server? Do you need to download the discogs data first?

That's correct.
The idea is simple: instead of requesting the data from the final site (discogs, etc) we request the data from a PHP script in our (portable, light) local webserver. The script will get the data from the final site and will give them back (after amending first off course). Check the "discogs new.src" attachment.

There are 2 parts.
1. IndexUrl
As you already know and mentioned API 1.0 gives more info vs API 2.0. On top of that API_KEY is not neded in API 1.0 (YES it is true).
The SearchBy parameter can be enriched now with other tags. Actually every tag/attribute under <release> tag (status, type, title, year, label, format, artist, etc) in API response is filtered from input parameter(s).

2. AlbumUrl
Here API 2.0 is really useful.
The XML is parsed and amended and then returned back.

Have you seen that discogs has a new API (v2.0) and the old one will be shut down in December?
http://forums.mp3tag.de/index.php?showtopic=14385
http://blog.discogs.com/2011/06/api-v20.html
I hope that doesn't affect you script.

IndexURL uses API 1.0 because master id is returned instead of all related versions (releases).
Still with a small addition I could request all versions from all master ids found. Maybe will make the response slower but it can be activated by a new parameter in the searchBy.

Please find 3 attachments with step-by-step setup.
the readme file first.
PHP local WEBserver is <40MB in total, portable (no registry, etc) and fully customizable.
I do have a NAS with PHP server pre-installed but the portable server works like a charm.

DetlevD has put in dispute local WEB server security but I don't find a risk there especially if pc is connected to a local router prior to the internet router (like I do).

edit [16/10/2011]:
PHP scripts are organized in class model now.
Version 1.0.0 is released...

I am working on some MusicBrainz PHP scripts too.

This post has been edited by Victor Kostas: Oct 16 2011, 14:27

Attached File(s)

 discogsVSK_v1.0.0.zip ( 13.83K ) Number of downloads: 34
 discogsVSK_Artist___otherIDs.src ( 7.61K ) Number of downloads: 52

 

[Victor Kostas]

PHP script has some more advantages.
If can combine several searches. E.g. Musicbrainz release does not include directly any artist roles (composer, etc). Instead an additional search must be done. Similar for bandmembers and other song attributes.

It would be nice if .src AlbumParse code is simplified. I.e. all tagnames/tagvalues are provided in "normalized" format from PHP. Then .src doesn't have to bother for a tag source (discogs, musicbrainz, etc).
But it looks like this is technically not feasible now. That would need to provide dynamically (from search result) <tagname> in

CODE

outputto <tagname>
say+++ command (sayuntil, sayregexp, sayrest, etc)
say "|"

Still an alternative would be using if <hardcoded tagname> statement. E.g.

CODE

do
findinline "<"
if "albumartist>"
outputto albumartist
findinline ">"
sayuntil "</albumartist>"
endif
if "albumartistsort>"
outputto albumartistsort
findinline ">"
sayuntil "</albumartistsort>"
endif
....
if "composer>"
outputto composer
findinline ">"
sayuntil "</composer>"
endif
while something

[Victor Kostas]

The local PHP web server could help on:

• Filtering artist releases by ANY tag or attribute.
  e.g. let's say we want to display only "461 Ocean Boulevard" CD releases of eric clapton.
  API 1.0 discogs url is http://www.discogs.com/artist/eric clapton?releases=1&f=xml
  The local server based script would be
  http://127.0.0.1:5723/discogsVSK/discogsVS...amp;artist=eric clapton&title=461 Ocean Boulevard&format=CD
  Tags queried are (artist, title, format).
  Other tags can be filtered are: Label, Year, id (release id), status (accepted, etc), type (main, etc).
  No need to amend PHP script for other tags can be returned from API.
  
  Thus PONE's web source scripts for AlbumArtists could be based on XML format.
  
• Executing multiple XML queries.
  e.g. API 2.0 artist query does not list all releases of master releases but only the main one. Master releases could be extended by additional API queries and return finally mp3tag web source script a single XML.
  
• Amending/restructuring final XML
  XML stream could be even restructured to help mp3tag web source script.

[pone]

Hi, I'm answering because you pointed me to this.

I don't want to discourage you, but I have to say that your local php web server project is far to complicated for me. I'm reading this topic with some interest, but I can't follow all the technical details.

Moreover, I don't see what you want to achieve by this? Is it just for being able to filter the discogs database in some way.
And do your filters work better than my different search scripts? I have to say that I have really no problems finding discogs entries with my script, I even added a new one for listing master release page entries.
You could easily add some more scripts to my script, based on discogs advanced search: http://www.discogs.com/advanced_search
You can use all tags wich have a search box there, and you don't have to limit in on maximal two tag fields at one search, as my script does at the moment. And you can also use tag field specific search in the standard serach script when you use this keywords: catno:, format:, artist:, label:, country:, track:, style:, as described here:
http://www.discogs.com/help/browsing-and-searching.html