Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

Filters for personalised radio #25

#1

Hello,

just getting started using mopidy and Rompr so apologies if these are silly questions

I usually use Kodi to play my music, but I have been unable to get it to start without a monitor on particular ubuntu server (normally use the libreelec variant on raspberry pis which works fine without a monitor) so after a bit of searching I came across mopidy and your code.

I mainly use kodi in "party" mode which plays random music continously that i can filter with rules, so I was particularly pleased to see your personalised radio station option.

Is there any way I can apply rules / filters to the song selections in the personal radio station to only

a) only pick a particular genre say jazz, folk or christmas for example
b) do not pick from a particular genre say christmas
c) only pick a particular artist
d) do not pick a particular artist
e) length of track say shorter that 4 minutes or longer than 10 minutes
f) pick a particular year
g) combine multiple rules together for example play jazz by miles davis where each track is longer than 12 minutes

etc etc

Thanks for creating a great tool

Tim

  • replies 37
  • views 11.2K
  • likes 0
tkgafs · Author
#2

Hi done a bit more digging and noticed the two choices for personal radios for genre and tracks by artists, although they seem to only select the first 100 or so entries and then choose five tracks from them so the effect seems to be the same few tracks occur quite often.

I've also had a look at tracktable in the database and notice it doesnt include Genre in it, is there a reason I'm not understanding as to why the genre is not in the database ?

I've added a column for genre into the Tracktable and amended the select statement in the "allrandom" case in metadatafunctions.php to test it
it seems to work fine for my needs which is only for playing localfiles, but I cant follow the code well enough to know if I will affect any other areas.

thanks again for the work you've done to produce this tool

tkgafs · Author
#3

diffs for this amendment

diff -r ../backends/sql/backend.php /var/www/rompr/backends/sql/backend.php
96c96
<                       (Title, Albumindex, Trackno, Duration, Artistindex, Disc, Uri, LastModified, Hidden, isSearchResult, Sourceindex, isAudiobook)
---
>                       (Title, Albumindex, Trackno, Duration, Genre, Artistindex, Disc, Uri, LastModified, Hidden, isSearchResult, Sourceindex, isAudiobook)
98,99c98,99
<                       (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
<               $data['title'], $data['albumindex'], $data['trackno'], $data['duration'], $data['trackai'],
---
>                       (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
>               $data['title'], $data['albumindex'], $data['trackno'], $data['duration'], $data['genre'], $data['trackai'],
1307a1308
>                       'genre' => $trackobj->tags['Genre'],
diff -r ../backends/sql/metadatafunctions.php /var/www/rompr/backends/sql/metadatafunctions.php
845a846,847
>                       // $sqlstring = "SELECT Uri FROM Tracktable WHERE Uri IS NOT NULL AND Genre LIKE 'Blues' AND Hidden=0 AND
>                       //      isSearchResult < 2";
diff -r ../backends/sql/sqlite/specifics.php /var/www/rompr/backends/sql/sqlite/specifics.php
42a43
>               "Genre TEXT, ".
57a59,63
>               } else {
>                       $err = $mysqlc->errorInfo()[2];
>                       return array(false, "Error While Checking Tracktable : ".$err);
>               }
>               if (generic_sql_query("CREATE INDEX IF NOT EXISTS ge ON Tracktable (Genre)", true)) {

now just to find a way to use it other than editing a file before choosing allrandom !!

#4

I thought I replied and then it disappeared.

Genre is not in the database, nor will it ever be. Personalised radio by Genre is a Music Discovery feature. When used with Mopidy and Spotify it'll play you tracks you've never heard before that aren't in your Collection (as well as tracks that are). It uses Mopidy's search to do this, otherwise I'd have to duplicate all of Spotify's database, so there's no need for Genre to be in Rompr's database.

The thing about it choosing only 100 or so entries sounds like a bug. Would you mind raising an issue over on GitHub so I don't forget. Not got any time for bug fixing right now.

tkgafs · Author
#5

thanks for reply

I dont use spotify last.fm etc and am only using mopidy as I couldnt get kodi to work without plugging in a monitor

I thought there had to be a reason why genre was not in the database

I'll raise an issue in github about the 100 entries

Tim

#6

GitQ keeps losing my replies. Apologies if you get this twice.

The 100 tracks limit is a limitation in Mopidy. I don't think MPD has that limitation (and if it does there's likely to be a configuration option) so you're better off with MPD. Genre radio and Artist radio both work with MPD.

As far as applying rules and filters, this isn't currently possible. I've been wanting to do something like a 'create your own custom radio station' feature for a while, but it's not easy - largely because I can't decide what the UI would look like. I'm open to suggestions.

tkgafs · Author
#7

Its worth looking at Kodi, it has a "party mode" which allows you to choose any rules you want to play random music

there is a simple screen allowing you to apply rules

gernre / artist / duration / year / album / title etc etc

contains / does not contain / equals / starts with etc /shorter than / longer than / year equals etc etc

rules can be stacked so you can end up with an extreme example like reggae from 1970 which is longer than 10 minutes
or Pop music which is shorter than 3 minutes and the album artist name is Various Artists

etc etc

currently installing MPD not sure where i got the idea it couldnt drive a local soundcard from

curiosly i found a couple of issues for mopidy discussing the limit issue

Capture.JPG

One thing to note if you switch to MPD, Rompr will not allow you to update your Music Collection, since it was created using Mopidy. You'll have to delete it (file is rompr/prefs/collection.sq3 if you're using the standard setup, or just drop the database and recreate it if you're using MySQL). If you have rating/tags/playcounts you want to keep you can do a Metadata Backup first and then restore it after re-creating the collection.

The Kodi UI looks similar to what I've envisioned in my head, which is both good and bad - means my ideas are on the right lines, but writing UIs like that is hugely tedious, which is mainly why I haven't done it yet :) Also making it do Genres is likely to mean I will have to put Genre in the database, which I've been trying to avoid.... but I'll have a good think about it.

If you're interested in checking out the current develop branch I've implemented a 'Create Custom Radio Station' feature that's works like Kodi's Party Mode. It's not quite finished, especially the UI, but it's functional and I'd be interested in feedback.

Note that if you try this (and in any case before you upgrade to the next release, whenever that comes out) you MUST revert the Genre change you made to your database BEFORE upgrading, as it will be incompatible with this update and will cause the new features to not work.

There are also new Genre and Artist options under 'Music From Your Collection' that use the database instead of search and so will return more than 100 tracks :)

Screen Shot 2020-03-02 at 15.27.28.png
tkgafs · Author
#10

I'll try it tonight, and let you know any feedback

tkgafs · Author
#11

In backends/sql/sqlite/Specifics.php It looks like you are not creating a column for genreindex in the tracktable if its a brand new database, code is there further down to alter tracktable to add column

#12

Holy moly, good spot.

#13

Should be fixed now. There might also be a problem with the default value being zero, I'm not sure, I'll have to inspect the code a bit tomorrow.

tkgafs · Author
#14

Wow great first cut of the code

A couple of inconsistencies

Parameters for Genre and Tracks by Artist are case sensitive
in the custom radio station the parameters are not case sensitive

Personally I prefer non case sensitive

One strange bug in the custom radio station selection

If I choose a specific year and either a genre or artist most of the time it works perfectly but occasionally it plays a track from the wrong year.
It seems to do this if an album has tracks from several different years and the first track on the album is the specific year requested, then it will play any tracks from that album as the station plays.

Addition criteria for the station rules I'd be grateful for

Album Artist (at the moment I cant search for tracks from compilations where the Album Artist is Various Artists for example)
Disc Number (useful for playing one disc from a multi disc set for example (loads of different ways of setting this tag though))
Track Title (play me all the different versions of a particular song for example (not sure how often this would be used by anyone though))

Being able to save the rulesets is excellent, dead easy to create decade type stations for example

I think you have already done virtually everything I would look for in a music player

Fantastic

tkgafs · Author

Yes as the year is only stored in the album table and seems to use the year of the first track, and the year is not stored in the track table there will always be issues with any rules that use years, if you have albums where the tracks are from different years ( I have lots of albums like this)

Its a never ending debate should an album be stored under original year of release, year of this actual release, different years in different countries etc etc

Personally I like every track to have the correct year against it, but thats just my view

I've just checked MPD and it does hold the correct year for each track in its database

#16

Thanks for the feedback, nice one.

Everything in Rompr should be case insensitive, so I’ll fix that.

The old debate about Year.... I’d never considered that albums could have tracks from different years but of course you’re right. Looks like that needs moving to the Tracktable. MPD uses the release date tag, which is usually set to ‘year of this actual release’, which I think is wrong but there ya go. Some taggers have an option to use year of original release for that field (I think Beets is one such) so I left it up to the user what data should be in there. I tend to agree with you though, I’ve got albums of 60s music where the year is set to 1980... not helpful :)

The other suggestions are all simple to do, so I can add those. I always like more options :)

#17

OK I added a bunch of new criteria. If you've created any saved Radios that use Artist you'll need to edit them again and choose either Track Artist or Album Artist, as the database query has changed.

Adding Year to the Tracktable is slightly more complicated than I first thought but it's doable. Just need to check through the code, some of the code in that area is old and not very well structured, and I've forgotten how it works..... :)

tkgafs · Author
#18

Just trying out the new criteria selections

There is an odd occasional issue if i select a specific year and a specific artist if that artist appears on an album which has a different album artist

I created a station with the following criteria year = 1975 artist = fairport convention

what was selected was the same track 5 times by the correct artist fairport convention but the year was 1969

The track was selected from an album where the Album Artist is set to Richard Thompson, the first track was the correct year 1975 and the last track was the correct Track Artist fairport convention although the year was wrong

A couple of screen shots may help explain

First the tags in the album

Capture.JPG

now the results of the radio staion

Capture2.JPG
tkgafs · Author
#19

How is Album Artist used in the selection criteria, it appears to select tracks where either the Artist or the Album Artist equals the selection criteria ?

rather than specifically Album Artist

tkgafs · Author
#20

The year selection criteria still seems locked to the first track on an album, then any track on that album will be chosen

tkgafs · Author
#21

Whoops just remembered you havent done much with year and track !!

#22

OK added Year to the Tracktable. The current code will need to update your Music Collection the first time you refresh the page.

If you've saved any Stations that use year, you'll need to re-edit them as the db query has changed again and if you use them as they are they'll still match on the album year.

How is Album Artist used in the selection criteria, it appears to select tracks where either the Artist or the Album Artist equals the selection criteria ?

That definitely should not be the case. If it still is - and you're sure you've re-edited any stations you've saved that use Artist, let me know.

tkgafs · Author
#23

All the year stuff seems to be working correctly as far as I have tested, what would be useful is a line somewhere saying how many songs have been found that match the criteria.

After you have performed the search are the found tracks randomised,

ie do you search for all matching tracks and randomise the results or

do you search for the first 5 tracks for the playlist and then search for each additional one as the playlist drops the just played one off the top

Ive had a couple of occasions where the same track has been played twice in a row

looking good

#24

The search is incremental; if I returned all tracks in one go I’d risk running out of memory in the browser if thousands were returned. Each track is flagged as having been used once it has been selected. If I try to find tracks and they’re all flagged as used then I reset all the flags and start again. Unfortunately I’ve no control over the database engine’s random function so I can’t fix the order. This could result in the same track being played twice in a row.

tkgafs · Author
#25

I've had a look at the Artist / Album Artist query as well and I understand what is happening not sure there is easy fix though, for a fairly esoteric use case anyway. Adding album artist index inside track table perhaps and seaching against that

I had expected doing a Search criteria for Album Artist would just return albums where the album artist is set to the search criteria

but in the db you record the albums artist to be the artist whether that has been found via either aritst or album artist tag

so a search for album artist will return tracks from all the artists albums

all the tracks within the album have the correct artist and the correct year

the issue is more my expectation of what the criteria would return rather than your coding !!

I have several assorted albums (the history of etc) where the album artist has been a member of various bands so on each individual track the artist tag is set to the correct artist and year of recording, but the overall album artist is set to the main person the collection is about, its really for sorting purposes I suppose as thats where I would file physical discs on a shelf

thanks for all your work on these new features

tkgafs · Author

the duplicates may also have been caused by me pressing the skip button quickly to scan through the traks checking to make sure they matched what I had searched for rather than actually listening to them !!

I have to be honest I love what you have done to this interface in a few days from my first tests, and I am seriusly thinking about replacing my kodi installations with it, even though I only originally wanted it for a special case where I couldn't use kodi

I had expected doing a Search criteria for Album Artist would just return albums where the album artist is set to the search criteria

That is exactly how it is supposed to work. 'Physical Discs on a shelf' is what the Collection is supposed to represent.

but in the db you record the albums artist to be the artist whether that has been found via either aritst or album artist tag

No I don't. It's complicated. The Albumtable has an AlbumArtisindex, which is the Album Artist. How that is set depends on how your files are tagged. If they are tagged with 'Album Artist' then I will use that. If they are not then I will try to decide on what the Album Artist should be - if more than 60% of the tracks on an album (an album being 'a bunch of files in the same folder') are by the same Artist (the 'Artst' tag), then that will be used as the Album Artist. If not then it will go under 'Various Artists', unless the number of tracks is too small to make an accurate guess, in which case I can't remember what happens :)

There's one case where this fails - in some cases I've found that MPD installations are configured to NOT return Album Artist tags. You can check this - go to the 'Debug Information' plugin in Rompr. Under 'Player Information' there is a 'tagtype' field. If 'AlbumArtist' is not in that list, then MPD is not giving me Album Artists. There are other tags that might also be used for Artist information (such as 'Sort Artist') but those are not supported and will be ignored.

If you look at your Music Collection, sorted by 'Artist', then those are the Album Artists that I'm aware of. If you're making a Radio that uses 'Album Artist' and it is playing tracks from albums that are not under that Artist in the Collection, then something is wrong.

#28

There are some difficult and alsmot-impossible-to-handle timing issues when clicking rapidly from one track to another and that could easily result in duplicates. You've got multiple things all happening asynchronously and simultaneously and I can't guarantee the order that things happen in. It's something I should look at for sure, but I don't see an easy way to cope with it, or indeed to test it.

I have to be honest I love what you have done to this interface in a few days from my first tests, and I am seriusly thinking about replacing my kodi installations with it, even though I only originally wanted it for a special case where I couldn't use kodi

Thanks. Once I get enthused about an idea I tend to run with it until I drop :)

tkgafs · Author
#29

There's one case where this fails - in some cases I've found that MPD installations are configured to NOT return Album Artist tags. You can check this - go to the 'Debug Information' plugin in Rompr. Under 'Player Information' there is a 'tagtype' field. If 'AlbumArtist' is not in that list, then MPD is not giving me Album Artists. There are other tags that might also be used for Artist information (such as 'Sort Artist') but those are not supported and will be ignored.

I cant see any options about Plugins or Debug Information anywhere, probably staring me in the face. I can turn on debug logging in setup page but thats the only place I can find, am I being really stupid.

albumartist tag is turned on in mpd config and its database contains the entries

I think the point I was making perhaps not very clearly was that you don't store Album Artist as a seperate column in the database if the tag exists, it is stored as the artist, therfore you cannot search for tracks on albums where an artist is defined as the album artist only, as you also get all the albums where they are the artist as well.

Its not a problem as it was only my wrong expectation of how you would use the tag, and its frankly a very low usage case

#30

Depends what skin you’re using but on the default Desktop skin the plugins all live in a menu you can access using the left-most icon above the Information Panel.

One of us is not understanding the other, that’s for sure ;) What you’re describing as how you think it should work is exactly the way it does work. So either there’s a misunderstanding or a bug, and I’d like to get to the bottom of that :)

Album Artist IS a separate column in the database. Every track has an Artist and an Album. Every album has an Album Artist. The Artist can be different. This being a database therefore, every track has an Artist and an Album Artist and they do not have to be the same.

Let’s use an example.

Suppose there is an Artist called Fred. You have some albums where the Album Artist is Fred. Fred also appears as an Artist on a track on an album where the Album Artist is called George. Searching for tracks by Album Artist Fred should return all of Fred’s albums, but not the track where Fred is the artist on George’s album. If that is not what you’re seeing then something is wrong somewhere. You should first check the Music Collection and make sure that the track where Fred is just an Artist appears under ‘George’ when the collection is sorted by Artist. If it is then the Album Artist is set correctly and the bug is somewhere in my new code. If it isn’t then you either have incorrect tags or the Collection hasn’t built as it is supposed to.

tkgafs · Author

Found the debug plugins, another feature I didnt know existed !!

it reports these tagtypes

tagtype Artist, Album, AlbumArtist, Title, Track, Name, Genre, Date, Composer, Performer, Disc

I think we are definitely speaking at cross purposes, most of my albums do not have the Album Artist tag set, but every tracks on every album has the correct artist set for the track, I only use Album Artist tags where there is a mixture of tracks by different artists or the artist is appearing in a slightly different lineup and I want to see the album listed with their other albums.

Suppose there is an Artist called Fred. You have some albums where the Album Artist is Fred. Fred also appears as an Artist on a track on an album where the Album Artist is called George. Searching for tracks by Album Artist Fred should return all of Fred’s albums, but not the track where Fred is the artist on George’s album. If that is not what you’re seeing then something is wrong somewhere. You should first check the Music Collection and make sure that the track where Fred is just an Artist appears under ‘George’ when the collection is sorted by Artist. If it is then the Album Artist is set correctly and the bug is somewhere in my new code. If it isn’t then you either have incorrect tags or the Collection hasn’t built as it is supposed to.

taking the above assume that on Freds albums the Album Artist tag is not set but each track has its artist set to Fred these will appear in the database under the artist index that matches Fred

but I have one album mainly with tracks by George and the artist tag for those tracks set to George but the ALbum Artist tag is Set to Fred for some reason or another

What I am saying is that there is no way to search for the Single Album where the Album Artist tag is set to Fred containing mainly tracks by George, because it'll have the same artistindex as all the albums by Fred which do not have the Album Artist tag set. To find this album would require another entry presumably in the tracktable where you currently have artistindex adding a seperate entry for albumartistindex.

the whole this is rather moot as I dont think it really matters as I doubt it would be used very much and the work you've done covers the major use of this, being able to search for compilation type albums using the Album Artist tag as its now set to Various Artists which you couldnt do at first.

#32

Right I get it. That most certainly is an edge case. You’re using Album Artist in brave and uncharted ways. :) You’ve got albums that you regard as having effectively no Album Artist. I designed Rompr’s collection specifically to cope with that scenario by assigning an Album Artist if the tag is not set, as I had a multitude of compilation albums that should be under Various Artists that no other player would sort correctly. In other words every album in Rompr has to have an Album Artist whether it exists as a tag or not.

The only way I could make your specific example work is if I added a field for album title, then you could search for album artist Fred and the album title. Adding Album Title as a new field is 3 lines of code so I can easily do it and therefore I will.

tkgafs · Author

Solution staring me in the face

I can say Album Artist is Fred and Artist is not Fred

this works almost perfectly, only omitting the tracks by Fred from the above album which is mainly by george

as you say adding a filter for album title would fix that case

Simples :)

tkgafs · Author
#34

Could the changes you have made had any afffect on updating the music collection

I added a couple of albums today and they appear correctly in the mpd tag_cache

if I search for the artist in rompr the albums appear but no artwork for them

if i go into the music collection and browse to the artist the albums do not appear in the list

if I go to update the artwork it says no albums are missing artwork even though they are !! they also do not show in the list of artists

In the database albumtable the items are there but the artwork fields are blank
the artist is also in the artiststable

I can also play the albums without problem

I have tried update the music collection now and Full local music rescan

All very strange

#35

It's possible, it could be to do with adding Genres.

First, your search for them will have added them to the database even if Update Music Collection Now didn't, so first, so we know what we're working with, we need to reset that. Search for something else, that will NOT match the new items.

Now, see if the new Albums and Artists are still in the database. If not, please enable debug logging at level 6, do Update Music Collection Now and send me the web server error log.

If the albums are still there, examine the Tracktable in the database. Check that the Tracks are there and the Albumindex is correct. Check that Hidden is NOT zero. Check that Genreindex is not NULL. Check that isSearchResult is 0 or 1. If any of those fields are wrong then I'll have to think some more.

#36

I found a typo in some of the new code which could explain it. Have tested it here, adding some new tracks, and it worked for me. Suggest you update to the latest develop branch and try again.

tkgafs · Author
#37

Yes that seems to have fixed issue, also noticed you have added the Album Title filter, once again thank you

#38

Thanks for you diligent testing. I've been wanting to implement something like this for ages but couldn't wrap my head around it until you pointed me at Kodi.