Analyze wikibase.fetchedEntities

Find a way to deal with the current global front-end data store.

• See bug 54082.

What are we talking about?

Problems

• Items with many statements result in a big serialized chunk. For example Q30 (United States of America) is 47 KiB, 7 KiB compressed. And the UI is slooow. :-(
• Not all data is needed by every widget.
• Other data is missing. Need to call the API anyway.
• No way to split/pagination.
• Not a good foundation for an UI that's different from the current.

Don't get me wrong

• The store is a cache. It avoids API calls.
• Fullfills all needs of the current UI.
• It's not actually broken. It works.
• Store + UI are a good solution for fast computers.

We need a master plan

Basic UI or rich UI first?

1. Basic UI (PHP + HTML) first?

• Strong foundation.
• Will cause many small requests in the beginning.
• Include more and more dynamic features for user-agents that support it.
• Save more and more requests. Optimize. Cache.
• Deploy often.

1. Rich UI (JavaScript) first?

• Promising in the beginning, possibly frustrating in the long run.
• Possibility to get stuck because of wrong decisions.
• You will build the same UI again for non-JS clients.

Ideas

• Use HTML/DOM as source (like a microformat).
• Split requests in chunks. Request only properties that are needed.
• Allow for nesting.
• Exclude parts that aren't needed, e.g. labels.
• Take care: To many small requests are also bad.
• Cache everything in localStorage.
• Create a nice API for the store.
• Store includes basic version control.
• Query via the store. Cached? Fast response. Outdated? Re-query automatically.
• Put changes in the store. The store updates the server.

How I look at big problems

• Why a big problem?
  • Huge codebase. 600 PHP and 150 JavaScript source files in the Wikibase repository alone.
  • Many widgets, all relying on the store.
• It works. No need to rush.
• I need to be familar with a code base.
• We need to talk. Make a list of all possible solutions, rate. Do something else. Days later: Still a good idea?
• I don't like breaking things. Optimize instead. Do small steps. Deprecate only if neccesarry.