EC makes idiotic plans for dangerous and unworkable mass online censorship

Copyright. Because - disappointingly, I couldn't find any good images for 'copyshite'

The European Commission has outlined plans for mandatory copyright filters on sites the public are allowed to post on. The filters will automatically decide whether some user-posted content - a video, say - infringes copyright and will automatically take down that content without input from any pesky, expensive humans.

This is the sort of thing Teresa May means when she talks about companies like Facebook using technology to take down 'extremist' content. But more on that later.

There are two problems with this idea. 

First, it is a clear violation of our human rights. These measures would constitute mass surveillance of all the content we post online and the automatic removal of the things we post that a company or government doesn't want us to.  This should worry you.  Governments and companies - however apparently benign - always want to censor complaints.  There's never been such a thing as a benign government and companies tend to get less so in direct proportion to the power they gain over their customers.  More worryingly still, governments are going to want access to the outputs of these filters. They're going to want to know who is posting what kind of content across multiple sites.  They'll want to know who's 'making trouble' by criticising party lines and what better way to do that by recognising what materials they've referenced in their works?  Companies will want access to the filters too and the ones running the filters will be all too happy to sell that access.  For a lot of companies, this kind of data would be pure gold. It would help them to better identify people as sales targets and to take down fair and protected criticism of their content.

Second, it won't work.  We know this because YouTube.  YouTube has spent years and several fortunes trying to solve this problem and the results are notoriously terrible.

Julia Redda is a German MEP and member of the Pirate Party Germany.  She wrote an article on (mostly) this second point. Filters of this sort don't work, have never worked and likely will never work, even if we wanted them to, which we shouldn't.

Here are some highlights:

5. Memory holes in the Syrian ArchiveAnother kind of filter in use on YouTube, and endorsed by the Commission, is designed to remove “extremist material”. It tries to detect suspicious content like ISIS flags in videos. What it also found and removed, however: Tens of thousands of videos documenting atrocities in Syria – in effect, it silenced efforts to expose war crimes.

I'll also wave vaguely in the direction of my point above about companies ravenously buying up access to filters. What better way to identify and censor whistleblowers?

6. Political speech removedMy colleague Marietje Schaake uploaded footage of a debate on torture in the European Parliament to YouTube – only to have it taken down for supposed violation of their community guidelines, depriving citizens of the ability to watch their elected representatives debate policy. Google later blamed a malfunctioning spam filter.

It was sweet of them to blame an oaf but really it was.... But ironically I can't find a clip of the Simpsons I wanted to link to to illustrate this. I wonder why.

7. Marginalised voices drowned outSome kinds of filters are used not to take down content, but to classify whether it’s acceptable to advertisers and therefore eligible for monetisation, or suitable for minors.
Recently, queer YouTubers found their videos blocked from monetization or hidden in supposedly child-friendly ‘restricted mode’ en masse – suggesting that the filter has somehow arrived at the judgement that LGBT* topics are not worthy of being widely seen.

Read the full article. And share it. But I think it's useful to put Redda's lessons supposedly but obviously not learned by the EC here:

• Lesson: That such a ridiculous error was made after years of investment into filtering technology shows: It’s extremely hard to get this technology right – if it is possible at all.
• Lesson: Copyright exceptions and limitations are essential to ensure the human rights to freedom of expression and to take part in cultural life. They allow us to quote from works, to create parodies and to use copyrighted works in education. Filters can’t determine whether a use is covered by an exception – undermining our fundamental rights.
• Lesson:Public domain content is at risk by filters designed only with copyrighted content in mind, and where humans are involved at no point in the process.
• Lesson: Automatic filters give big business all the power. Individual users are considered guilty until proven innocent: While the takedown is automatic, the recovery of an illegitimate takedown requires a cumbersome fight.
• Lesson: Filters can’t understand the context of a video sufficiently to determine whether it should be deleted.
• Lesson: There are no safeguards in place to ensure that even the most obviously benign political speech isn’t caught up in automated filters
• Lesson: Already marginalised communities may be among the most vulnerable to have their expression filtered away.
• Lesson: Legitimate, licensed uploads regularly get caught in filters.
• Lesson: Not even those wielding the filters have their algorithms under control.

Remember: all these lessons are illustrated in Julia Redda's article with examples. 

Remember: lawmakers are recommending forcing this kind of clusterfuck on millions of people without understanding that they don't work or caring that they violate our most basic human rights.

Remember: that Julia Redda and others (the EFF, for example) are fighting this and you can support them.

W3C makes idiotic decision, EFF resigns

This is terrible news. The World Wide Web Consortium (W3C) is the organisation that controls open standards for the web. It's mission has always been to protect the interests of ordinary web users as well as the corporate members of the W3C.  Today it abandoned the principle of empowering web users in favour of bowing to the pressure of corporate members acting in their own interests. The web is set to become a worse place because of this.

As Cory Doctorow writes at Boing Boing:

In July, the Director of the World Wide Web Consortium overruled dozens of members' objections to publishing a DRM standard [for video] without a compromise to protect accessibility, security research, archiving, and competition.

As a direct result, the Electronic Frontier Foundation (EFF) has resigned from the consortium. This is also very bad news indeed. The EFF has consistently had our collective interests at heart since its formation in 1990. Without their presence at the table, who knows what further havoc the W3C will wreak.

DRM (Digital Rights Management) is some electronic means by which content publishers can protect their content from being pirated. This would be perfectly acceptable - no reasonable person wishes to deprive artists or software writers of their rightful income - if it weren't for the fact that the model is deeply flawed.  DRM usually means that we don't, in fact, own the movie or the music or the software we just bought. We own a license to use that content, a license which can be taken away from us by the company if they don't like what we're doing with it.

This means that companies can prevent us from lending music or books to other people. They can stop us moving content to different media so we can back it up or play it in different circumstances (on our phones, from a CD, etc) It means they can summarily cut access to our content or stop our computers from working if they decide we've violated their terms and conditions. It means that they can stop us from using third-party ink in our printers because they want us to buy the more expensive kind from them. And it means they can artificially inflate the price of that ink in the knowledge that we'll have to pay.

In these days of the Internet of Shit Things, where all manner of devices in our lives - from light bulbs to cars - are internet connected, this is especially troubling. If manufacturers can turn off our cars if they think we're voiding the warranty or can tie us into mandatory deals with 'approved' insurance companies (who might also be able to turn off the car if they don't like the way we're driving or the places we're driving to) then we're pretty much fucked.

This is not an outlandish scenario. Some vehicle manufacturers already make it virtually impossible for a third party (or the owner) to service and fix their own vehicles because DRM prevents access to diagnostic tools.  This is made worse by bewilderingly terrible laws around the world which make it illegal to disrupt or bypass DRM.  For example, the Digital Millennium Copyright Act in the US makes bypassing DRM a federal offense. Modifying the software on a John Deere tractor in the US, for example, so that you can diagnose and fix a problem, can land you a five year prison sentence.

Not only do you not own your music and books, you don't own your car, tractor or many of the other things you think you own. They can be taken away from you on a whim and there's not a great deal you can do about it.

Adopting DRM as a web standard is especially troubling. It means that browsers get to decide how you can use content based on the arbitrary whim of the consortia providing that content. This has dire implications in a number of important areas:

• Security researchers might not be permitted to break DRM to investigate whether products are safe. Users won't know whether the services they're using are secure and companies will likely be disincentiveised from ensuring their products are safe.
• Archiving of important content will be impossible (without breaking DRM which is illegal in some countries and very likely to become so in many more very soon). The grief of losing a great work of art because of the selfish interest of the rights owner is tragic. The outrage of losing documentaries of atrocities and the true version of history is an abomination. That governments will seek to do so if they have the means is simply a given.
• Accessibility will be compromised. For example, subtitling of videos is essential to those with hearing difficulties. 300 hours of video are uploaded to YouTube every minute. Automatic subtitling is the only way to cope with this volume and that would involve breaking DRM.
• The big media and software companies reached their position through innovation: they provided new platforms that circumvented established companies who would otherwise have been their competitors. Adopting DRM as a web standard will prevent new companies from doing the same thing, stifling competition.

Read the article for more details, but I quote the EFF's letter of resignation from the W3C in full below. I have a feeling that the EFF won't mind.

The W3C's decision is due to pressure from its big consortium members. It has allowed the interests of a small number of rich people to run rough-shod over the interests of billions of internet users. This is the exact opposite of the freedoms it was established to protect. It wouldn't surprise me if its founder and current director, Tim Berners-Lee, found himself being appointed to the board of various member consortia in the near future. The most disappointing part of all this is that the W3C doesn't seem to understand (or understands but doesn't care) that it holds the power. What are the powerful consortium members going to do? Build a new web?

The EFF's campaign against DRM as a web standard has been exploratory. It has suggested compromises which would have given the pro-DRM everything they want (or at least, everything they claim to want) while protecting the rights of users.

Cory Doctorow again:

EFF appealed the decision, the first-ever appeal in W3C history, which concluded last week with a deeply divided membership. 58.4% of the group voted to go on with publication, and the W3C did so today, an unprecedented move in a body that has always operated on consensus and compromise. In their public statements about the standard, the W3C executive repeatedly said that they didn't think the DRM advocates would be willing to compromise, and in the absence of such willingness, the exec have given them everything they demanded.

They "didn't think" the pro-DRM partners would be willing to compromise so instead of forcing them to compromise or taking the motion off the table entirely, they decided to screw us all.

Here's the EFF's resignation letter:

Dear Jeff, Tim, and colleagues, 

In 2013, EFF was disappointed to learn that the W3C had taken on the project of standardizing “Encrypted Media Extensions,” an API whose sole function was to provide a first-class role for DRM within the Web browser ecosystem. By doing so, the organization offered the use of its patent pool, its staff support, and its moral authority to the idea that browsers can and should be designed to cede control over key aspects from users to remote parties.

When it became clear, following our formal objection, that the W3C's largest corporate members and leadership were wedded to this project despite strong discontent from within the W3C membership and staff, their most important partners, and othersupporters of the open Web, we proposed a compromise. We agreed to stand down regarding the EME standard, provided that the W3C extend its existing IPR policies to deter members from using DRM laws in connection with the EME (such as Section 1201 of the US Digital Millennium Copyright Act or European national implementations of Article 6 of the EUCD) except in combination with another cause of action.

This covenant would allow the W3C's large corporate members to enforce their copyrights. Indeed, it kept intact every legal right to which entertainment companies, DRM vendors, and their business partners can otherwise lay claim. The compromise merely restricted their ability to use the W3C's DRM to shut down legitimate activities, like research and modifications, that required circumvention of DRM. It would signal to the world that the W3C wanted to make a difference in how DRM was enforced: that it would use its authority to draw a line between the acceptability of DRM as an optional technology, as opposed to an excuse to undermine legitimate research and innovation.

More directly, such a covenant would have helped protect the key stakeholders, present and future, who both depend on the openness of the Web, and who actively work to protect its safety and universality. It would offer some legal clarity for those who bypass DRM to engage in security research to find defects that would endanger billions of web users; or who automate the creation of enhanced, accessible video for people with disabilities; or who archive the Web for posterity. It would help protect new market entrants intent on creating competitive, innovative products, unimagined by the vendors locking down web video.

Despite the support of W3C members from many sectors, the leadership of the W3C rejected this compromise. The W3C leadership countered with proposals — like the chartering of a nonbinding discussion group on the policy questions that was not scheduled to report in until long after the EME ship had sailed — that would have still left researchers, governments, archives, security experts unprotected.

The W3C is a body that ostensibly operates on consensus. Nevertheless, as the coalition in support of a DRM compromise grew and grew — and the large corporate members continued to reject any meaningful compromise — the W3C leadership persisted in treating EME as topic that could be decided by one side of the debate.  In essence, a core of EME proponents was able to impose its will on the Consortium, over the wishes of a sizeable group of objectors — and every person who uses the web. The Director decided to personally override every single objection raised by the members, articulating several benefits that EME offered over the DRM that HTML5 had made impossible.
But those very benefits (such as improvements to accessibility and privacy) depend on the public being able to exercise rights they lose under DRM law — which meant that without the compromise the Director was overriding, none of those benefits could be realized, either. That rejection prompted the first appeal against the Director in W3C history. 

In our campaigning on this issue, we have spoken to many, many members' representatives who privately confided their belief that the EME was a terrible idea (generally they used stronger language) and their sincere desire that their employer wasn't on the wrong side of this issue. This is unsurprising. You have to search long and hard to find an independent technologist who believes that DRM is possible, let alone a good idea. Yet, somewhere along the way, the business values of those outside the web got important enough, and the values of technologists who built it got disposable enough, that even the wise elders who make our standards voted for something they know to be a fool's errand. 

We believe they will regret that choice. Today, the W3C bequeaths an legally unauditable attack-surface to browsers used by billions of people. They give media companies the power to sue or intimidate away those who might re-purpose video for people with disabilities. They side against the archivists who are scrambling to preserve the public record of our era. The W3C process has been abused by companies that made their fortunes by upsetting the established order, and now, thanks to EME, they’ll be able to ensure no one ever subjects them to the same innovative pressures.
So we'll keep fighting to fight to keep the web free and open. We'll keep suing the US government to overturn the laws that make DRM so toxic, and we'll keep bringing that fight to the world's legislatures that are being misled by the US Trade Representative to instigate local equivalents to America's legal mistakes.
We will renew our work to battle the media companies that fail to adapt videos for accessibility purposes, even though the W3C squandered the perfect moment to exact a promise to protect those who are doing that work for them.
We will defend those who are put in harm's way for blowing the whistle on defects in EME implementations. 

It is a tragedy that we will be doing that without our friends at the W3C, and with the world believing that the pioneers and creators of the web no longer care about these matters. 

Effective today, EFF is resigning from the W3C. 

Thank you,
Cory Doctorow
Advisory Committee Representative to the W3C for the Electronic Frontier Foundation

Guy misses point about privacy, at least doesn't claim to be cyborg

A rail company replaces tickets with a chip in your hand. The conductor scans the chip with a phone (presumably it's an NFC chip). The phone identifies the customer from the ID number stored on the chip then looks in the database to find out whether the customer has paid for the journey.

It means you can't lose your ticket, I guess, but it leaves a trail which is hackable by criminals and governments and is presumably subject to court orders from law enforcement agencies. It's not for me, but I won't judge people who find the convenience is worth all the leaking privacy. And I have to admit that this kind of thing is still pretty cool, even though it's generally a bad idea.

There's an almost criminal lack of detail in this short video (which was filmed for the Travel Show of all things), but there's enough to get the general idea. The WARNING WILL ROBINSON part comes where the presenter asks the guy from the company about privacy. He says it's OK because the chip doesn't transmit and that it only contains a customer ID so nobody "outside the company" will be able to find out anything about you.

Yeah, that's all bollocks. Of course the chip transmits, otherwise how could the phone read it? What he means is that the chip doesn't have a power source so it only transmits when an NFC reader (presumably such as the one on my phone) is in close proximity and powers up the chip itself.  Surely there's no way to scan someone's chip without their knowing, right?  Yeah, and pockets never get picked.

But that's not the main issue. The company guy is arguing that the ID number is not very useful by itself because all the information about the customer is in a database somewhere else, which is assumed to be secure.

It isn't. Someone will get at it sooner or later. Let's look at how this system might work:

Let's give the company the benefit of the doubt and say that the conductor's phone contains only a list of IDs of the customers who have paid for a particular journey. That would be the most secure option and the only thing the conductor needs to know. Live access by the phone to the company's database, for example, would be a very bad idea for lots of reasons. In the interest of brevity, I won't cover them here (ask in the comments if you're interested).

But that list will need to be updated before and during the journey. This could happen via wifi or mobile networks. So perhaps the train has a server which is constantly updated by a mobile signal and the phone talks to the server by wifi. Or maybe the phone itself is directly updated by a mobile service. It doesn't matter what scenario is used, these connections are all points at which attacks are very attractive. The chances of there being a way to attack those points of vulnerability are not particularly slim.

Once someone has got into that system, there's a good chance that mischief will be possible. Even if the network interface strictly implements the rule that customers are referred to only by their ID numbers, what's the betting that the company that supplies the kit and software won't be able to resist bundling extra features? What if the customer wants a physical receipt to claim expenses? Will the conductor be able to print one? In which case the transaction and customer data will need to be on the air at some point, where it is vulnerable. We can think of many other scenarios where the customer's data would not be 'safely' behind a firewall.

And that's before someone has managed to get hold of the conductor's phone, in which case all security bets are off.

But all of these potential attacks are a needlessly elaborate way to get access to sensitive personal and travel information. It's all right there in a server somewhere, ripe for stealing. Hackers, unfriendly governments (domestic and foreign) and law enforcement agencies would be able to get access to all the customer information and the journeys they'd made by hacking the company's servers or getting a court order. Ill-intentioned people within the company might also be able to get this data. It's not as though that sort of thing doesn't happen all the time.

Why would this matter? Here are a few scenarios:

1. This is data you don't want to get into the hands of stalkers or abusive exes. I know of at least one case where an abusive ex-husband got hold of his ex-wife's location via her lease car's built-in tracking system (which was there to disable it in case she stopped making payments, itself a security threat). Or what if the conductor took a shine to a customer and abused the system to find out where they lived, what other journeys they make and so on?
2. Blackmailers (who might be criminals, governments etc) could search for unusual journeys by brute force and might, with a tiny amount of additional detective work, find targets.  They could also determine to some degree which customers might have been traveling together.
3. Law enforcement agencies might search the database to find targets for further inquiry, putting everyone who traveled on a particular route under suspicion. Once someone is a target of suspicion, they might be subject to additional scrutiny and when mining data from several sources, it's hard not to find a fit, even if it's not for the crime under investigation.
4. People will be able to tell whether you actually traveled on that train. You may buy a paper ticket for someone else. You might send an e-ticket to their phone. A ticket is a transferable document, even if it isn't physical. But a chip-ticket is not. I might buy a ticket for someone fleeing violence, for example, so that person could not be traced by their persecutors. But not if it's a chip-ticket.

I don't mind the fact that many people will find this a convenient way to pay although I wouldn't encourage it (especially if you're up to something). What bothers me is the blasé attitude of the guy from the company who is lying to the BBC when he says there isn't a privacy issue. He goes on to spout the usual nonsense about unlocking your car and house and so on. Yeah it can be done, but yeah it's a stupid way to do it.

The shop where you pay with privacy

Kaspersky Lab has opened a pop-up store in London. It doesn't accept money, though, you pay with
private data.  It's a publicity stunt, of course, but a good one.  People have been 'buying' stuff too, even though Kaspersky usually gives this stuff away for free.

The prices of item varies:

[...]to acquire the mug, you had to hand over three photos, or screenshots of your WhatsApp, SMS and email conversations, to Kaspersky. To buy the t-shirt, it had to be the last three photos on your Camera Roll — so you couldn't be selective — or the last three messages on your phone. The original print, finally, forced you to hand over your phone. A member of staff would then poke around and select five photos or three screenshots. 

What's interesting is how punters are reacting:

There was a mixture of excitement and nervousness in the store. Some people were caught off guard and immediately started rummaging through their phones, checking photos and messages for anything that might cause embarrassment. [...] "But [a member of staff said] when you're actually asked to exchange this private information and walk away with something that does have monetary value, people are like, 'Whoa! What is actually on my telephone? What are the messages that I've sent?' It's a little bit scary."

The reporter, Nick Summers, had originally intended to go for the original print, handing over his phone for staff to poke around in. He panicked, though and went for a mug instead (choosing three pictures to disclose). I don't blame him: I am abnormally well aware of what's on my phone because I'm paranoid about that sort of thing, but I'd certainly hesitate before handing anyone my phone.

What I like about this is that the customers are obviously willing participants, playing along because they want to experience the panic of willfully exposing private data.  They're learning about privacy through role play, which is a great idea.

How to trust?

Trust is an enormously important part of security. How and why we trust things is the basis for both security in principle and in practice. Ultimately, we have to trust our password managers, for example, and - more importantly - the services we sign up for. I know that Amazon is using my data to dubious and potentially harmful purpose. I trust that my VPN provider is not.

That trust might well be misguided. It's based partly on reputation and partly on a vague belief that businesses probably won't harm their long-term future to make a fast buck. This is not a safe assumption, even in the VPN business. Due to the mistrust most citizens of most countries currently have for their government, VPN companies have sprung up all over the place. Quite a lot of them are fraudulent, either not actually providing a proper VPN service or sucking up and exploiting your data willy-nilly.

We live in an age where the US president has shown us that he can break the rules everyone thought were there to limit the power of presidents.... and nobody will do a thing about it.

We live in an age where trusted retail pharmacists such as Boots in the UK sell quack medicines alongside actual medicines with no way to tell the two apart.

So trust is important, all the more so because it can be so easily subverted when people just don't play by the rules we expect them to play by.  We need to establish better bases for trust and one of those is reputation. We look at the reviews. We're all pretty savvy, right? We look at the negative reviews first and use them to establish a kind of context for the positive reviews. If the negative reviews are all about late delivery or the occasional breakage or that the buyer clearly ordered the wrong thing in the first place, we are more comfortable accepting the positive reviews.

We do that because glowing reviews are easy to manufacture and bad ones are more difficult to fake.  And we do it because we tend to think we can recognise fake reviews of either type.  We're probably kidding ourselves.

Cory Doctorow writes:

One of the reasons that online review sites still have some utility is that "crowdturfing" attacks (in which reviewers are paid to write convincing fake reviews to artificially raise or lower a business or product's ranking) are expensive to do well, and cheap attacks are pretty easy to spot and nuke. 

But in a new paper, a group of University of Chicago computer scientists show that they were able to train a Recurrent Neural Network (RNN) to write fake reviews that test subjects could not distinguish from real reviews -- and moreover, subjects were likely to rate these as "helpful" reviews.

The bad news is that neural nets can write convincing reviews. The good news is that other neural nets are even better at spotting fake reviews. The worse news is that they need our metadata to do this, the most obvious being our social graphs. Presumably anonymous reviews will do little to train these neural nets to recognise fake reviews, further reducing any basis we have for trust.

Those of us who work to put metadata back into our own hands might be inadvertently harming everyone by making trust ever harder to come by.

Fridge Porn

I've never bought into the idea that the fridge is somehow a family hub and therefore should be part Internet of Shit. I don't even understand why families need a hub. Everyone in every family I know has at least one internet-connected device and is contactable at pretty much any time. Why should family members have to come home and stare at the fridge to find out that they should have bought milk while they were out?

And for that matter, what's wrong with bits of paper and magnets?

But that aside, here's a story of the inevitable.  Remember back in the 80s when electronics stores had things we called 'home computers' that we could play with?  Who didn't go in there and write something like:

10 PRINT "FUCK OFF "

20 GOTO 10

on every BBC Micro they had?

Nobody, that's who.

Here is the modern day equivalent. Someone drew dick pics on sticky notes on a fridge in a Home Depot shop and left porn streaming in a browser. Because of course they did, who wouldn't?

This was discovered when the organization’s head toured a visitor through the office, and wanted to show off a streaming feature on the Samsung fridge.

Fantastic.