IPB

Welcome Guest ( Log In | Register )

2 Pages V  < 1 2  
Reply to this topicStart new topic
> Discogs: Parse Album (Release)
DetlevD
post Apr 12 2011, 15:13
Post #16


Member


Group: Full Members
Posts: 5031
Joined: 26-May 06
From: Wuppertal, Germany, Planet Earth
Member No.: 3194
Mp3tag Version: 2.63



QUOTE (Victor Kostas @ Apr 12 2011, 14:05) *
... First priority is to be able to convert Discogs XML file with less hustle for user. ...

Could you please elaborate this idea in more detail?

DD.20110412.1613.CEST


--------------------
* Beyond that, don't ask, when you don't know what to do with the answer. *
♥ home is where the heart is ♥
Go to the top of the page
 
+Quote Post
Victor Kostas
post Apr 12 2011, 15:49
Post #17


Member


Group: Full Members
Posts: 96
Joined: 2-March 08
From: Belgium
Member No.: 6578
Mp3tag Version: 2.65



QUOTE (DetlevD @ Apr 12 2011, 16:13) *
Could you please elaborate this idea in more detail?

DD.20110412.1613.CEST


Currently, implemented Web source scripts are split in 2 steps:
1. Parse candidate releases (ParseIndex). Discogs provides HTML results but no XML ones.
2. Parse selected release (ParseAlbum). Discogs provides both HTML and XML results.

Thanks to your suggestion (use Local Web Server) I came with one idea: AlbumURL (Parse no 2) to not request Release directly from Discogs but from local Web Server instead. Local server will request the release info (XML format) from Discogs. The XML will be amended and then returned to mp3tag Web Source. A PHP script must be implemented in the local Web Server.
Nothing is changed in the way Web source works.

There are plenty Local Web Servers in the net. I prefer portable versions (no installation required).
I used XAMPP package which contains Web Server, PHP, FileZilla and others.

What do you think?
Go to the top of the page
 
+Quote Post
DetlevD
post Apr 12 2011, 16:48
Post #18


Member


Group: Full Members
Posts: 5031
Joined: 26-May 06
From: Wuppertal, Germany, Planet Earth
Member No.: 3194
Mp3tag Version: 2.63



QUOTE (Victor Kostas @ Apr 12 2011, 16:49) *
... I came with one idea: AlbumURL (Parse no 2) to not request Release directly from Discogs but from local Web Server instead. Local server will request the release info (XML format) from Discogs. The XML will be amended and then returned to mp3tag Web Source. A PHP script must be implemented in the local Web Server.
Nothing is changed in the way Web source works.
There are plenty Local Web Servers in the net. I prefer portable versions (no installation required).
I used XAMPP package which contains Web Server, PHP, FileZilla and others.
What do you think?

I think ... well ... today many people are permanently linked together via several networks and services ... and they all have open doors to their harddisks ... but you cannot demand it from a normal user that he agrees to the installation of a web server on his home computer.
There might be a simpler way.
No extra web server, just a simple HTTP requester.

Maybe of interest ... there is a source where different access methods and tools are described ...
http://techsupt.winbatch.com/webcgi/webbat...h~Web~Pages.txt
Could be XMLStarlet of some use?

DD.20110412.1748.CEST

This post has been edited by DetlevD: Apr 12 2011, 16:48


--------------------
* Beyond that, don't ask, when you don't know what to do with the answer. *
♥ home is where the heart is ♥
Go to the top of the page
 
+Quote Post
Victor Kostas
post Apr 12 2011, 16:59
Post #19


Member


Group: Full Members
Posts: 96
Joined: 2-March 08
From: Belgium
Member No.: 6578
Mp3tag Version: 2.65



QUOTE (DetlevD @ Apr 12 2011, 17:48) *
I think ... well ... today many people are permanently linked together via several networks and services ... and they all have open doors to their harddisks ... but you cannot demand it from a normal user that he agrees to the installation of a web server on his home computer.
There might be a simpler way.
No extra web server, just a simple HTTP requester.

Maybe of interest ... there is a source where different access methods and tools are described ...
http://techsupt.winbatch.com/webcgi/webbat...h~Web~Pages.txt
Could be XMLStarlet of some use?

DD.20110412.1748.CEST


I will have a look. Thanks.
In general my goal is to implement an intermediate script between (discogs) XML and mp3tag Web source script Album parser. Thus, album XML parser script will be simplified too.
Of course installation of local web server is a disadvantage.
Thank you again for your findings.
Go to the top of the page
 
+Quote Post
Victor Kostas
post Oct 4 2011, 20:27
Post #20


Member


Group: Full Members
Posts: 96
Joined: 2-March 08
From: Belgium
Member No.: 6578
Mp3tag Version: 2.65



I have finished (a beta version) of 2 PHP scripts which will help a lot with discogs site XML data.
1st script is used to filter releases of one given artist. Currently discogs returns all artist releases in a big XML without giving the option of any kind of filter... Well until now.

2nd script is "normalization" of discogs release XML.
Distribute Extraartists in track level.
Calculate artistmulti for each artists tag.
Calculate composer, lyricist, involvedartistst in track level.

With those 2 scripts we can deal only with (amended) XML files. Scripts can be easily amended to keep mp3tag webscripts simple.

With a local PHP server it is pretty easy.
I will come with more details.
Any thoughts?
Go to the top of the page
 
+Quote Post
pone
post Oct 4 2011, 20:40
Post #21


Member


Group: Full Members
Posts: 1688
Joined: 15-March 09
From: Germany
Member No.: 9103
Mp3tag Version: 2.59b



So your script can only be used if you have a local php server? Do you need to download the discogs data first?

Have you seen that discogs has a new API (v2.0) and the old one will be shut down in December?
http://forums.mp3tag.de/index.php?showtopic=14385
http://blog.discogs.com/2011/06/api-v20.html

I hope that doesn't affect you script.
Go to the top of the page
 
+Quote Post
Victor Kostas
post Oct 4 2011, 21:31
Post #22


Member


Group: Full Members
Posts: 96
Joined: 2-March 08
From: Belgium
Member No.: 6578
Mp3tag Version: 2.65



QUOTE (pone @ Oct 4 2011, 21:40) *
So your script can only be used if you have a local php server? Do you need to download the discogs data first?

That's correct.
The idea is simple: instead of requesting the data from the final site (discogs, etc) we request the data from a PHP script in our (portable, light) local webserver. The script will get the data from the final site and will give them back (after amending first off course). Check the "discogs new.src" attachment.

There are 2 parts.
1. IndexUrl
As you already know and mentioned API 1.0 gives more info vs API 2.0. On top of that API_KEY is not neded in API 1.0 (YES it is true).
The SearchBy parameter can be enriched now with other tags. Actually every tag/attribute under <release> tag (status, type, title, year, label, format, artist, etc) in API response is filtered from input parameter(s).

2. AlbumUrl
Here API 2.0 is really useful.
The XML is parsed and amended and then returned back.

QUOTE (pone @ Oct 4 2011, 21:40) *
Have you seen that discogs has a new API (v2.0) and the old one will be shut down in December?
http://forums.mp3tag.de/index.php?showtopic=14385
http://blog.discogs.com/2011/06/api-v20.html
I hope that doesn't affect you script.

IndexURL uses API 1.0 because master id is returned instead of all related versions (releases).
Still with a small addition I could request all versions from all master ids found. Maybe will make the response slower but it can be activated by a new parameter in the searchBy.

Please find 3 attachments with step-by-step setup.
book.gif the readme file first.
PHP local WEBserver is <40MB in total, portable (no registry, etc) and fully customizable.
I do have a NAS with PHP server pre-installed but the portable server works like a charm.

DetlevD has put in dispute local WEB server security but I don't find a risk there especially if pc is connected to a local router prior to the internet router (like I do).

edit [16/10/2011]:
PHP scripts are organized in class model now.
Version 1.0.0 is released...

I am working on some MusicBrainz PHP scripts too.

This post has been edited by Victor Kostas: Oct 16 2011, 14:27
Attached File(s)
Attached File  discogsVSK_v1.0.0.zip ( 13.83K ) Number of downloads: 71
Attached File  discogsVSK_Artist___otherIDs.src ( 7.61K ) Number of downloads: 141
 
Go to the top of the page
 
+Quote Post
Victor Kostas
post Oct 9 2011, 12:45
Post #23


Member


Group: Full Members
Posts: 96
Joined: 2-March 08
From: Belgium
Member No.: 6578
Mp3tag Version: 2.65



PHP script has some more advantages.
If can combine several searches. E.g. Musicbrainz release does not include directly any artist roles (composer, etc). Instead an additional search must be done. Similar for bandmembers and other song attributes.

It would be nice if .src AlbumParse code is simplified. I.e. all tagnames/tagvalues are provided in "normalized" format from PHP. Then .src doesn't have to bother for a tag source (discogs, musicbrainz, etc).
But it looks like this is technically not feasible now. That would need to provide dynamically (from search result) <tagname> in
CODE
outputto <tagname>
say+++ command (sayuntil, sayregexp, sayrest, etc)
say "|"


Still an alternative would be using if <hardcoded tagname> statement. E.g.
CODE
do
findinline "<"
if "albumartist>"
outputto albumartist
findinline ">"
sayuntil "</albumartist>"
endif
if "albumartistsort>"
outputto albumartistsort
findinline ">"
sayuntil "</albumartistsort>"
endif
....
if "composer>"
outputto composer
findinline ">"
sayuntil "</composer>"
endif
while something

Go to the top of the page
 
+Quote Post
Victor Kostas
post Oct 16 2011, 23:27
Post #24


Member


Group: Full Members
Posts: 96
Joined: 2-March 08
From: Belgium
Member No.: 6578
Mp3tag Version: 2.65



The local PHP web server could help on:
  • Filtering artist releases by ANY tag or attribute.
    e.g. let's say we want to display only "461 Ocean Boulevard" CD releases of eric clapton.
    API 1.0 discogs url is http://www.discogs.com/artist/eric clapton?releases=1&f=xml
    The local server based script would be
    http://127.0.0.1:5723/discogsVSK/discogsVS...amp;artist=eric clapton&title=461 Ocean Boulevard&format=CD
    Tags queried are (artist, title, format).
    Other tags can be filtered are: Label, Year, id (release id), status (accepted, etc), type (main, etc).
    No need to amend PHP script for other tags can be returned from API.

    Thus PONE's web source scripts for AlbumArtists could be based on XML format.
  • Executing multiple XML queries.
    e.g. API 2.0 artist query does not list all releases of master releases but only the main one. Master releases could be extended by additional API queries and return finally mp3tag web source script a single XML.
  • Amending/restructuring final XML
    XML stream could be even restructured to help mp3tag web source script.

Go to the top of the page
 
+Quote Post
pone
post Oct 17 2011, 01:55
Post #25


Member


Group: Full Members
Posts: 1688
Joined: 15-March 09
From: Germany
Member No.: 9103
Mp3tag Version: 2.59b



Hi, I'm answering because you pointed me to this.

I don't want to discourage you, but I have to say that your local php web server project is far to complicated for me. I'm reading this topic with some interest, but I can't follow all the technical details.

Moreover, I don't see what you want to achieve by this? Is it just for being able to filter the discogs database in some way.
And do your filters work better than my different search scripts? I have to say that I have really no problems finding discogs entries with my script, I even added a new one for listing master release page entries.
You could easily add some more scripts to my script, based on discogs advanced search: http://www.discogs.com/advanced_search
You can use all tags wich have a search box there, and you don't have to limit in on maximal two tag fields at one search, as my script does at the moment. And you can also use tag field specific search in the standard serach script when you use this keywords: catno:, format:, artist:, label:, country:, track:, style:, as described here:
http://www.discogs.com/help/browsing-and-searching.html
Go to the top of the page
 
+Quote Post

2 Pages V  < 1 2
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 31st October 2014 - 23:55