get_iplayer cannot get metadata since 13th March (Monday afternoon)

Hello group.

I've not been able to update the index since Monday afternoon.
Requesting the full TV listing (piped to a text file) reveals an index
with less than 7000 entries, none later than about 2pm last Monday.

If I try to force a list of *added* entries by looking for "channel
patrol", it comes up empty. It looks like either something's screwed up
my installation of get_iplayer (seems unlikely to me but I can't be
totally certain) or else the Beeb have screwed something up at their end,
either by accident or on purpose.

Before I start trying to fix a possibly non existent problem and risk
another breakage event (I'm running Linux Mint 17.1), I thought I'd best
ask here whether anyone else using GiP has been experiencing the same
problem.

I first noticed this issue on the Monday evening. Hoping it was just a
temporary hiccup at the Beeb's end, I just grabbed the PIDs from the
Beeb's TV schedule pages one by one to feed into the get_iplayer command
line. This worked but it reported it wasn't able to access the metadata
leading to filenames made up from the PID (in duplicate) alone.

Luckily, this is a mere nuisance rather than a show stopper since I
record the SD broadcasts off air using kaffeine's recording scheduler to
provide both a backup against undownloadable iplayer offerings as well as
to provide a cue for when iplayer content becomes available for download
and my normal practice is to tag the date and episode number and title to
the epg derived filename of each off-air recording (eg "The Hunt 4-7
20170314 Hunger at Sea (Oceans).m2t" where the 4-7 stands in for 4/7
meaning episode 4 of 7, the 20170314 obviously being the broadcast date
coded as yyyymmdd and the trailing text being the episode name - not all
series offer episode names or, in some cases, even an episode number!).

Once an iplayer file has been downloaded and (now) converted into an mp4,
I simply copy and paste the name of the m2t version and prepend the date
code with a hash symbol to indicate that the file is an iplayer sourced
version. Once I've verified that I have good iplayer versions of the off-
air m2t recordings, I delete the m2ts and put the mp4s into longer term
storage on the NAS box.

Tonight (Tuesday), I hit the same problem again, indicating that this
might be a more permanent change of the status quo, hence my seeking help
here before I start breaking things in a futile attempt to remedy what
may well be a problem of the Beeb's own making rather than an issue at my
end of the process.

Normally, I'd get GiP to generate an index list of "Added" programmes
since the last update which keeps this abridged list small enough to fit
into the Konsole's history buffer and make it more relevant for that
day's broadcasts. I then simply pencil the relevant index numbers onto a
small notepad which I then type into the get_iplayer command line.

It's a relatively trivial process (compared to my PID work-around during
the current crisis) which allows me to deal with other stuff whilst
get_iplayer processes each entry one by one until the list is exhausted.

The last time I tried to get get_iplayer to process a list of PIDs, it
either failed miserably or only processed the first one (it was quite a
while back and, pragmatic as ever, once I had a solution that worked, I
didn't bother pursuing any other alternatives).

I know this might seem to be a bit too "Hands On" but most of the
programmes I'm recording with Kaffeine are almost entirely exclusive to
the BBC and not on a rigid schedule, so, apart from "Eggheads", I tend to
trawl Kaffeine's epg listings every few days to manually add what takes
my fancy to the recording schedule (at least I don't have to agonise over
channel or padding clashes). Therefore, it was only natural to use a
similarly pragmatic approach to fetching iplayer content with get_iplayer.

The question, basically right now, is this:-

Is this a general problem for everyone using get_player or just me?

--
Johnny B Good

On Wed, 15 Mar 2017 05:21:07 +0000, Johnny B Good wrote:

SNIP..............

Is this a general problem for everyone using get_player or just me?

It is not just you:

https://squarepenguin.co.uk/forums/announcement-20.html

No our accessible I player app for pc for the blind written by the Web Ie
folk with the cooperation of the bbc I player team seems to have failed
also.
What on earth are they doing now. If its not broken why fix it and break
other stuff.

Words fail me really.
Brian

--
----- -
This newsgroup posting comes to you directly from...
The Sofa of Brian Gaff...

Blind user, so no pictures please!
"Johnny B Good" wrote in message
...
Hello group.

I've not been able to update the index since Monday afternoon.
Requesting the full TV listing (piped to a text file) reveals an index
with less than 7000 entries, none later than about 2pm last Monday.

If I try to force a list of *added* entries by looking for "channel
patrol", it comes up empty. It looks like either something's screwed up
my installation of get_iplayer (seems unlikely to me but I can't be
totally certain) or else the Beeb have screwed something up at their end,
either by accident or on purpose.

Before I start trying to fix a possibly non existent problem and risk
another breakage event (I'm running Linux Mint 17.1), I thought I'd best
ask here whether anyone else using GiP has been experiencing the same
problem.

I first noticed this issue on the Monday evening. Hoping it was just a
temporary hiccup at the Beeb's end, I just grabbed the PIDs from the
Beeb's TV schedule pages one by one to feed into the get_iplayer command
line. This worked but it reported it wasn't able to access the metadata
leading to filenames made up from the PID (in duplicate) alone.

Luckily, this is a mere nuisance rather than a show stopper since I
record the SD broadcasts off air using kaffeine's recording scheduler to
provide both a backup against undownloadable iplayer offerings as well as
to provide a cue for when iplayer content becomes available for download
and my normal practice is to tag the date and episode number and title to
the epg derived filename of each off-air recording (eg "The Hunt 4-7
20170314 Hunger at Sea (Oceans).m2t" where the 4-7 stands in for 4/7
meaning episode 4 of 7, the 20170314 obviously being the broadcast date
coded as yyyymmdd and the trailing text being the episode name - not all
series offer episode names or, in some cases, even an episode number!).

Once an iplayer file has been downloaded and (now) converted into an mp4,
I simply copy and paste the name of the m2t version and prepend the date
code with a hash symbol to indicate that the file is an iplayer sourced
version. Once I've verified that I have good iplayer versions of the off-
air m2t recordings, I delete the m2ts and put the mp4s into longer term
storage on the NAS box.

Tonight (Tuesday), I hit the same problem again, indicating that this
might be a more permanent change of the status quo, hence my seeking help
here before I start breaking things in a futile attempt to remedy what
may well be a problem of the Beeb's own making rather than an issue at my
end of the process.

Normally, I'd get GiP to generate an index list of "Added" programmes
since the last update which keeps this abridged list small enough to fit
into the Konsole's history buffer and make it more relevant for that
day's broadcasts. I then simply pencil the relevant index numbers onto a
small notepad which I then type into the get_iplayer command line.

It's a relatively trivial process (compared to my PID work-around during
the current crisis) which allows me to deal with other stuff whilst
get_iplayer processes each entry one by one until the list is exhausted.

The last time I tried to get get_iplayer to process a list of PIDs, it
either failed miserably or only processed the first one (it was quite a
while back and, pragmatic as ever, once I had a solution that worked, I
didn't bother pursuing any other alternatives).

I know this might seem to be a bit too "Hands On" but most of the
programmes I'm recording with Kaffeine are almost entirely exclusive to
the BBC and not on a rigid schedule, so, apart from "Eggheads", I tend to
trawl Kaffeine's epg listings every few days to manually add what takes
my fancy to the recording schedule (at least I don't have to agonise over
channel or padding clashes). Therefore, it was only natural to use a
similarly pragmatic approach to fetching iplayer content with get_iplayer.

The question, basically right now, is this:-

Is this a general problem for everyone using get_player or just me?

--
Johnny B Good

From that then it sounds like the bbc did not mean to break it, but then do
they ever test anything properly when they shove it out? Its been my
experience that they don't.
Load of amateurs.
Brian

--
----- -
This newsgroup posting comes to you directly from...
The Sofa of Brian Gaff...

Blind user, so no pictures please!
"Richard" wrote in message
o.uk...
On Wed, 15 Mar 2017 05:21:07 +0000, Johnny B Good wrote:

SNIP..............

Is this a general problem for everyone using get_player or just me?

It is not just you:

https://squarepenguin.co.uk/forums/announcement-20.html

On Wed, 15 Mar 2017 12:02:41 +0000, Brian Gaff wrote:

From that then it sounds like the bbc did not mean to break it, but then
do they ever test anything properly when they shove it out? Its been my
experience that they don't.
Load of amateurs.
Brian

That's an insult to 'amateurs'. However, that aside, "**** Happens" in
real life and I'm sure maintaining the iplayer servers is not immune to
this this particular 'fact of life'.

In this case, the fact that this particular example of excess manure had
still not been cleared up 24 hours after the initial dump (the evidence I
could see pointed to it happening around 2 or 3 pm on the Monday) was
becoming a cause for concern, the question being whether this was just an
accident or a deliberate change in the system requiring yet another GiP
update or a fancy work-around.

Richard's posting of the link to the get_iplayer Support Forums is
appreciated and I now have that bookmarked. According to their latest
announcement, the iplayer metadata sources have now been restored. I
haven't taken the opportunity to test this claim just yet since I
normally wait to the end of the evening before triggering another refresh
to generate a list of "Added" index numbers which list is more relevant
and also compact enough to fit into Konsole's history buffer (saves me
piping the output of the get_iplayer --tv command into a half MB or so
text file).

Assuming the announcement is true (I've no reason to believe otherwise)
and they haven't managed to break it yet again, then I expect to confirm
in a little under 3 hours time that, to quote the punchline of a Bob
Monkhouse joke, "Norman Jervis has been exhumed." :-)

--
Johnny B Good

On Wed, 15 Mar 2017 03:39:33 -0500, Richard wrote:

On Wed, 15 Mar 2017 05:21:07 +0000, Johnny B Good wrote:

SNIP..............

Is this a general problem for everyone using get_player or just me?

It is not just you:

https://squarepenguin.co.uk/forums/announcement-20.html

Ah, thanks very much for that, Richard. As I mentioned in my follow up
to Brian's post, I've yet to test the truth of that announcement but I
expect it'll all be back to "Normal Service" when I do my next grab,
around or just before midnight.

--
Johnny B Good

On Wed, 15 Mar 2017 21:26:17 +0000, Johnny B Good wrote:

On Wed, 15 Mar 2017 03:39:33 -0500, Richard wrote:

On Wed, 15 Mar 2017 05:21:07 +0000, Johnny B Good wrote:

SNIP..............

Is this a general problem for everyone using get_player or just me?

It is not just you:

https://squarepenguin.co.uk/forums/announcement-20.html

Ah, thanks very much for that, Richard. As I mentioned in my follow up
to Brian's post, I've yet to test the truth of that announcement but I
expect it'll all be back to "Normal Service" when I do my next grab,
around or just before midnight.

It all seems to be back to normal now (now being about 1am Thursday).

And, it looks like it's staying that way (around 1 am today) so it would
appear that "Norman Jervis has been exhumed" in regard of the iplayer
service.

--
Johnny B Good