>>! In T66214#2782449, @Anomie wrote:
> I think redirects are probably //essential//. You say that the performance impact of redirects as a mechanism to encourage clients to update their code is bad, but throwing fatal errors at them is far, far worse.

Where did I state that throwing a fatal error was the alternative? That's nonsense.

> Sure, put a Varnish layer in front of the API as deployed for WMF to canonicalize the URLs as much as possible so the API itself doesn't have to return redirects. [[https://phabricator.wikimedia.org/T149408#2759693|But don't make a complicated Varnish layer be required for the API to be easily usable by clients of low-traffic installations.]]

Nothing in what I proposed would make the URL scheme for small installations problematic. We're talking about improving cache fragmentation at the Varnish level for Varnish itself. On smaller wikis by definition the image set isn't gigantic and should they set up a caching layer the way we do with Varnish, fragmentation probably wouldn't be the end of the world. And that's assuming misbehaving clients on top of small wikis. Small wikis are quite likely to either be using well-supported extensions, which by default would probably do the right thing after this migration happens, or their own code, in which case they are responsible for their own sensible behavior when it comes to cache fragmentation at the backend caching layer. If anything the flexibility of a query string-based solution makes things easier for small wikis, as the logic behind the scheme is more universal and much less WMF-specific than what we currently have.

>>! In T66214#2782563, @Anomie wrote:
> Not necessarily. A naive query-string builder based on a python dict or a perl hash, for example, is liable to have a random parameter ordering. Perl, in particular, [[http://perldoc.perl.org/perlsec.html#Algorithmic-Complexity-Attacks|intentionally randomizes the order of hash entries for security reasons]].

That's something developers using those languages should be aware of. We can't design a URL scheme or API defensively, based on terrible defaults in language X or Y. It's those languages problem, not ours. And it's probably the cause of cache fragmentation on many APIs, if constructing URLs with the helper functions in those languages results in random parameter ordering with given values. And while the example you give is for hashes, do functions in those languages that construct URLs really do that too, given who counter-productive that obviously is for caching?


>>! In T66214#2782454, @GWicke wrote:
> Maintaining all kinds of specific parameter validation and rewrite rules in Varnish, and keeping them in sync with the actual backend service does not sound very practicable. If we do our job of designing APIs right, then we shouldn't need to resort to such band-aids. 

That's wishful thinking, because designing a URL scheme now can only be based on what you know now. You can't predict the future and unforeseen needs will come up that will directly conflict with an overly broad policy like "we can't introduce new parameters with defaults on old urls". The current URL scheme is testament that future needs are difficult to anticipate. Look at the history of the existing URL scheme and plan for the obvious: it will have to change and it will need massaging for backwards compatibility. If you don't plan for that, we'll get stuck again with a new format that has an equally painful migration plan as the current one. The inflexibility and hard rules need to go away, because that's exactly the source of how painful change in that area is at the moment.

So far the only blocker to having a fully flexible format seems to be cache fragmentation. We can solve Varnish cache fragmentation, we just need to figure out how far that might have to go once we go down that rabbit hole. We can solve it at the client level by propagating best practices. It could be as simple as issuing a warning in headers of the response when a client doesn't have sorted parameters, in addition to documentation and communication.

>>! In T66214#2782454, @GWicke wrote:
> This is because any thumbnail can have any number of author-supplied parameters already

Can you provide a concrete example? I don't think I'm familiar with that feature.