1080i vs 1080p

On Fri, 21 Jun 2019 17:13:35 +0100, Roger Wilmut wrote:

On 2019-06-20 10:58:04 +0000, Scott said:

Scott wrote:

with 625 lines you could sometimes see the "mice teeth" but with 1080
I never notice them.

If you want the official name it's 'combing' and occurs when someone has
moved between the odd scan and the even scan (or vice versa). In HD
because the lines are so fine it just looks like blurring. You wouldn't
get it with 1080p (full scan 50 times per second) as you get with
Blu-Ray, but as said above it eats up twice the bandwidth which isn't
really practicable.

With modern video display technology, those distinctions, progressive
and interlaced display modes, no longer seem to apply (at least not in
the sense of 50p using twice the bandwidth of 25p).

Obviously 50fps interlaced is a rather cheeky way to halve the bandwidth
requirements over 50fps progressive, best applied dynamically when rapid
changes would normally distract the viewers' minds from attempting to
analyse the scene in detail. This subterfuge is quite effective as far as
meeting the demands of the target audience goes.

Obviously, for someone determined to examine the less frenetic parts of
the scene, they'll discover the evidence of such trickery readily enough.
However, such close analysis isn't really a fair test since few of us
would be foolish enough to emulate this behaviour in real life by
ignoring the action in the immediate environment just to admire the
detail they can perceive by concentrating on the patterning in the tarmac
of a busy road whilst crossing it.

However when it comes to 50 versus 25 fps progressive, the 50p generally
consumes a few percent less than the 25p version as produced by
Handbrake's compression of the iplayer 2.3GB per 59 minute 1280x720 50fps
ts downloads into 50 fps "H264 - MPEG-4 AVC (part 10)(avc1)" which can
range in size from a low of 850MB to a high of 1.8GB (and without any
loss of quality over get_iplayer's ffmpeg conversions to mp4).

Relying upon get_iplayer's use of ffmpeg to translate the 2.3GB ts files
into 2.2GB mp4 does rather imply that 50fps material consumes twice the
bandwidth of the 25fps 1.1GB per 59 minute 1280x720 iplayer downloads
that are no longer an option (it's now either SD 960x540 25fps or 50fps
1280x720 unless, for very often inexplicable reasons, there's only the SD
version or worse on offer).

Before iplayer withdrew the 25fps option for the 1280x720 half HD
versions, I was quite happy to let get_iplayer deal with the .ts to .mp4
conversion with ffmpeg since it produced consistent 1010 to 1020MB sized
mp4 files for almost every 1.1GB ts download of a 59 minute programme
(the "Never mind the quality, feel the width!" to serving up 'content'
approach taken at iPlayer Towers) whereas using Handbrake to convert
these ts files into h264 mkv files generally resulted in file sizes
ranging from 1.2 to 1.8GB (as I'd experienced back in the days before I'd
upgraded ffmpeg to the version get_iplayer had demanded).

When I was faced with this choice of 50fps only versions of the 1280x720
or else put up with a 25fps 960x540 SD version which had more than
doubled the size of these iplayer downloads, I tried to shrink them back
down to their original 1GB per hour sizes for storage using Handbrake to
transcode/compress them into 25fps H264 mkv files.

This did indeed offer (on average) a reduction to half the downloaded
size but with the variation as mentioned above. However, there would
occasionally be the rare exception where the resulting file landed up
being slightly larger than the download which led me to experiment with
the "Keep the same framerate as the original" option which led to the
rather astonishing finding that almost all such 50fps h264 mkvs would be
1 or 2 percent smaller than the 25fps transcodes!

Once I'd made that discovery, it was 50fps mkvs from then on (except, of
course, when my only option would be a 960x540 25fps **** poor
substitute).

In the intervening 18 months or so since the 25fps 1280x720 versions
were dropped from iPlayer's download/streaming offerings, I've had plenty
of time to ponder this repealing of the law that states that 50fps
progressive format video shall require twice the bandwidth of a 25fps
progressive format video.

It seems to me that the reduction in frame to frame differences at 50fps
is virtually cancelling the larger but less frequent frame to frame
differences in a 25fps version to the extent that a 50fps version
requires no more, indeed often a little less, bandwidth than the 25fps
version provided an efficient codec algorithm such as h264 is employed to
take fuller advantage of the additional redundancy hidden within a 50fps
video stream.

Handbrake's compression to h264 is clearly taking every opportunity to
eliminate redundancy within the video streams it encodes since programmes
that could be described as "Content Free" such as "Eggheads", "Mock The
Week" and HIGN4U, which are recorded against a largely static studio
backdrop result in file sizes of around 350MB versus the earlier 500 or
so MB mp4s created by get_iplayer.

Knowing that the odd 59 minute programme landing up as a 1.2 to 1.6GB mkv
simply means there was more 'video content' than the more usual 8 to 9
hundred MB's worth, I no longer fret over the space being consumed on the
server compared to the old "One size fits all" mentality behind iPlayer's
"Never mind the quality, feel the width!" 2.3GB per 59 minute programme
download of their 50fps 1280x720 offerings.

When this "2.3GB per 59 minute 50fps 1280x720 programme" nonsense was
imposed by the gyppos running the iPlayer concession stand some 18 months
back, the time penalty was a mere 3 minutes extra on a 3 minute download.
However, over the past 6 to 9 months, the iPlayer servers have been
progressively throttling the download speeds from the 50 to 60Mbps it had
previously been until, for the past month or so, it's now at a paltry 5
to 7 Mbps resulting in download times that are now slightly slower than a
real-time download (1 hour 5 minutes ETA for a 59 minute programme).

For the past few months I've resorted to running get_iplayer in
concurrent Konsole sessions at the end of each evening. In some cases,
I've had as many as 9 downloads on the go, each running at the throttled
speed of 5 to 6 Mbps I get regardless of how many are running
concurrently, indicating that it's nothing to do with the speed of my VM
supplied service unless VM have illegally imposed some weird traffic
shaping policy in order to sell me their own media services. Apropos of
which, is anyone else using get_iplayer experiencing a similar speed
throttling?

--
Johnny B Good

On 22/06/2019 23:44, Johnny B Good wrote:

However, over the past 6 to 9 months, the iPlayer servers have been
progressively throttling the download speeds from the 50 to 60Mbps it had
previously been until, for the past month or so, it's now at a paltry 5
to 7 Mbps

Apropos of
which, is anyone else using get_iplayer experiencing a similar speed
throttling?

I wonder if this is a time of day thing? I am currently downloading a
film that was on BBC last night and get_iplayer averaged 25Mbps down a
VM 50Mbps broadband cable.

Jim

"Mark Carver" wrote in message
...

The whole reason the BBC chose to have the dynamic p/i switching on
HD DTT,

Is it switched and/or flagged by the broadcaster? I'd assumed 25p
content was just autodetected by the TV.

is because the de-interlacer in their encoder is (or should be !)
miles better than the one in everyone's telly.

1080 50p isn't supported on Freeview HD DTT, only 50i or 25p, and I'm
unaware of any broadcaster deinterlacing 50i to 25p.

The deinterlacers used by broadcasters (for other purposes) are
relatively uncomplicated; they give mediocre but consistent results
across all content types.

On Sun, 23 Jun 2019 12:24:35 +0100, Indy Jess John wrote:

On 22/06/2019 23:44, Johnny B Good wrote:

However, over the past 6 to 9 months, the iPlayer servers have been
progressively throttling the download speeds from the 50 to 60Mbps it
had previously been until, for the past month or so, it's now at a
paltry 5 to 7 Mbps

Apropos of which, is anyone else using get_iplayer experiencing a
similar speed throttling?

I wonder if this is a time of day thing? I am currently downloading a
film that was on BBC last night and get_iplayer averaged 25Mbps down a
VM 50Mbps broadband cable.


Lucky you! I rarely see iplayer downloads go faster than 6Mbps these
days (or nights - ToD makes no difference). This evening I saw speeds in
the 5.23 to 5.44 Mbps range. It doesn't matter how many concurrent
downloads are involved, they all run within about +/-5% of each other
whether it's just two or as many as nine or ten on the go.

Unless VM are applying some very clever and selective (yet completely
illegal) throttling to each download stream, it does look very much like
they're being throttled at iPlayer Towers.

Thank you for sharing that information btw. It's a useful comparison
data point, suggesting that I'm being targetted for 'special treatment',
either by my isp or the iPlayer servers. I guess it's time to check out
get_iplayer's command line switch options to see whether I can use a uk
proxy to hide my isp provided IP address.

I've just located the proxy commands but the use of a proxy assumes you
already have a proxy connection set up and ready to go. The closest
dealings I've ever had with any proxy was the use of Privoxy in win2k
over a decade ago, so this will be a learning curve for me.

I guess my main problem will be finding a free proxy service that's both
trustworthy and unknown to the iplayer servers. It looks like I've got
some research to complete before I can go any further in solving this
mysterious bandwidth throttling of my iPlayer downloads.

--
Johnny B Good

On 23/06/2019 20:46, Alex wrote:
"Mark Carver" wrote in message
...

The whole reason the BBC chose to have the dynamic p/i switching on
HD DTT,

Is it switched and/or flagged by the broadcaster? I'd assumed 25p
content was just autodetected by the TV.

No, the broadcaster's encoder detects, and changes mode accordingly.
When the Beeb started doing it, some TV receivers didn't change modes
seamlessly, and they (the manufacturers) had to provide patches

Read all about it:-

https://www.tomshardware.co.uk/BBC-HD-1080p-Freeview-HD-Settop-Boxes,news-35640.html


1080 50p isn't supported on Freeview HD DTT, only 50i or 25p, and I'm
unaware of any broadcaster deinterlacing 50i to 25p.

Read the links provided, when the encoder detects no difference between
odd and even fields in a 50i signal, it switches (on a GOP by GOP level)
to 25p


--
Mark
Please replace invalid and invalid with gmx and net to reply.

Mark Carver wrote:

When the Beeb started doing it, some TV receivers didn't change modes
seamlessly, and they (the manufacturers) had to provide patches

My samsung stores picture settings (brightness/contrast/saturation etc)
individually for interlaced and progressive picture types, which led to
to horrid picture flashes when the picture type changed every few
seconds mid-stream ...

On 24/06/2019 12:20, Andy Burns wrote:
Mark Carver wrote:

When the Beeb started doing it, some TV receivers didn't change modes
seamlessly, and they (the manufacturers) had to provide patches

My samsung stores picture settings (brightness/contrast/saturation etc)
individually for interlaced and progressive picture types, which led to
to horrid picture flashes when the picture type changed every few
seconds mid-stream ...

Yes, and even after Sony fixed the audio glitch problem on the 2010/2011
models, the firmware still forces you to have to set the overscan on/off
separately in p and i mode (and you can only make the relevant setting
when it's actually in the relevant mode)

--
Mark
Please replace invalid and invalid with gmx and net to reply.

"Mark Carver" wrote in message
...
On 23/06/2019 20:46, Alex wrote:

Is it switched and/or flagged by the broadcaster? I'd assumed 25p
content was just autodetected by the TV.

No, the broadcaster's encoder detects, and changes mode accordingly.
When the Beeb started doing it, some TV receivers didn't change
modes seamlessly, and they (the manufacturers) had to provide
patches

Read all about it:-

https://www.tomshardware.co.uk/BBC-HD-1080p-Freeview-HD-Settop-Boxes,news-35640.html

Fair enough - although if it's based purely on "automatic pulldown
detection" as it's sometimes called, there are likely to be mistakes
made, even with the best detector. I thought perhaps the programme
makers might flag the source material, similar to the way AFDs worked.



1080 50p isn't supported on Freeview HD DTT, only 50i or 25p, and
I'm
unaware of any broadcaster deinterlacing 50i to 25p.

Read the links provided, when the encoder detects no difference
between odd and even fields in a 50i signal, it switches (on a GOP
by GOP level) to 25p

Sorry to be pedantic, but you specifically referred to the BBC's
deinterlacer being better as their reason for having dynamic p/i
switching on HD. De-interlacing and pulldown detection are of course
two different things, which is why I misunderstood your post.

On 24/06/2019 15:37, Alex wrote:

Fair enough - although if it's based purely on "automatic pulldown
detection" as it's sometimes called, there are likely to be mistakes
made, even with the best detector. I thought perhaps the programme
makers might flag the source material, similar to the way AFDs worked.

That was my thought too when they started doing it, but it is totally
automagic. That said, I have noticed on BBC News Channel HD it does
sometimes drop into 25p during the weather forecast if the presenter
stops moving. Notably this is normally the only time the news ticker
isn't present.
It's likely the ticker is enough to stop the encoder dropping into p
mode, even if the rest of the picture material is 100% suitable.

Read the links provided, when the encoder detects no difference
between odd and even fields in a 50i signal, it switches (on a GOP
by GOP level) to 25p

Sorry to be pedantic, but you specifically referred to the BBC's
deinterlacer being better as their reason for having dynamic p/i
switching on HD. De-interlacing and pulldown detection are of course
two different things, which is why I misunderstood your post.

Don't worry, no problem sir.


--
Mark
Please replace invalid and invalid with gmx and net to reply.

On Mon, 24 Jun 2019 03:06:46 +0000, Johnny B Good wrote:

On Sun, 23 Jun 2019 12:24:35 +0100, Indy Jess John wrote:

On 22/06/2019 23:44, Johnny B Good wrote:

However, over the past 6 to 9 months, the iPlayer servers have been
progressively throttling the download speeds from the 50 to 60Mbps it
had previously been until, for the past month or so, it's now at a
paltry 5 to 7 Mbps

Apropos of which, is anyone else using get_iplayer experiencing a
similar speed throttling?

I wonder if this is a time of day thing? I am currently downloading
a
film that was on BBC last night and get_iplayer averaged 25Mbps down a
VM 50Mbps broadband cable.


Lucky you! I rarely see iplayer downloads go faster than 6Mbps these
days (or nights - ToD makes no difference). This evening I saw speeds in
the 5.23 to 5.44 Mbps range. It doesn't matter how many concurrent
downloads are involved, they all run within about +/-5% of each other
whether it's just two or as many as nine or ten on the go.

Unless VM are applying some very clever and selective (yet completely
illegal) throttling to each download stream, it does look very much like
they're being throttled at iPlayer Towers.

Thank you for sharing that information btw. It's a useful comparison
data point, suggesting that I'm being targetted for 'special treatment',
either by my isp or the iPlayer servers. I guess it's time to check out
get_iplayer's command line switch options to see whether I can use a uk
proxy to hide my isp provided IP address.

I've just located the proxy commands but the use of a proxy assumes you
already have a proxy connection set up and ready to go. The closest
dealings I've ever had with any proxy was the use of Privoxy in win2k
over a decade ago, so this will be a learning curve for me.

I guess my main problem will be finding a free proxy service that's
both
trustworthy and unknown to the iplayer servers. It looks like I've got
some research to complete before I can go any further in solving this
mysterious bandwidth throttling of my iPlayer downloads.

Mystery Solved!

It turns out the culprit proved to be my Netgear GS608 v2 8 port Gbit
ethernet switch becoming flakey (most likely down to 'blown' caps but
I've yet to find a 'tear down' video to show me how to break into the
box).

During the past few weeks I'd started seeing the usual 60 to 70 MB/s
write speeds for data transfers from my Linux Mint desktop to the NAS4Free
box hovering around the 30 to 40 MB/s mark which I'd put down to Linux's
rather abysmal SMB performance becoming just a little more abysmal than
usual[1]. I suppose the downgrade must have started around the time I'd
started to see the drop in iPlayer download speeds but the effect on LAN
only transfers was rather more subtle so I didn't immediately see any
link.

What confounded the possibility of seeing a common theme between LAN
only and internet speeds was the fact that I could run multiple downloads
from iPlayer's servers without any apparent effect on the speeds of the
individual streams involved. This held true to the extent that I could
aggregate a full 70Mbps's worth of streaming before any slowdown effect
became apparent on the individual streams. This apparent throttling of
individual downloads was so consistent, it looked very much like the
result of a deliberate throttling policy at iPlayer Towers.

It was only this afternoon that I finally started to suspect a problem
with the NAS LAN connection after seeing worrying symptoms accessing the
disk volumes and folders during the past week or two, including writing
speeds to the NAS dropping below the 20MB/s into the MB/s region for
several seconds at a time. I finally put two and two together to pick on
the most likely culprit, the rather hot running GS608 switch.

Fortunately the GS608's predecessor, a GS605, was still in my stock
cupboard unsold so I was able to effect an immediate swap out with a new,
hardly used 5 port Gbe switch which immediately restored the status quo.
Luckily, due to reduced demands as the inevitable result of "Empty Nest
Syndrome", the loss of three LAN ports is no longer an issue. :-)


[1] A test with an up to date windows 7 machine a few years back had
shown that the BSD based NAS4Free box had been capable of 125MB/s
transfers since its last MoBo upgrade a few years earlier (about ten
years ago now) until the local caches had filled/emptied dropping down to
the 85 to 95MB/s limiting speeds of the HDDs involved at each side of the
gigabit ethernet link.

Prior to that discovery, I'd been under the misapprehension that the
60MB/s write/55MB/s read speeds between my win2k desktop and the NAS had
been a NAS box issue (which made the reduction of LAN speeds to 25MB/s
when booting the NAS hardware from a Knoppix Live CD as a temporary work
around to the EXT2 FS driver 1TiB address wrap around issue, all the more
telling of Linux's abysmal SMB data transfers performance).

A word to the wise who wish to roll their own NAS box setup, is to avoid
a Linux solution and go for a NAS4Free/FreeNAS or other FreeBSD based
solution instead if you wish to support SMB data transfers at full Gbe
speeds.

--
Johnny B Good