Discovery Snapshot 2015-12-31 02:00:10

Done 27 (27 story points)
Backlog 10 (10 story points)
In progress 5 (5 story points)
Needs review 1 (1 story points)
Resolved 152 (152 story points)
total 195 (195 story points)
Title Priority Story Points Assignee Status
#122617 repoint oozie job failure emails to discovery-alerts@wikimedia.org Needs Triage 1 - Backlog
#122425 codfw has six unassigned shards Normal 1 EBernhardson In progress
#121832 Completion suggester beta feature is missing an RTL logo Normal 1 EBernhardson Done
#121547 Improve Language Identification Training Data via Application of Language Models to the Training Data Needs Triage 1 - Backlog
#121543 Do an A/B Tests on Other Wikis with TextCat for Language Identification Needs Triage 1 - Backlog
#121542 Do an A/B Test on enwiki using TextCat for Language Identification Needs Triage 1 - Backlog
#121541 Create Properly Weighted Language Identification Evaluation Sets for Top N Other Wikis Needs Triage 1 - Backlog
#121540 Investigate Updating Cybozu / ES Plugin for Language Identification Needs Triage 1 dcausse Done
#121539 Create Balanced Language Identification Evaluation Set for Top N Wikis by Query Volume High 1 - Backlog
#121538 Convert TextCat to PHP Library for Language Identification in Cirrus Search High 1 Smalyshev In progress
#121376 test and validate completion suggester beta feature on beta cluster Needs Triage 1 Deskana Done
#121375 build completion suggester indices on beta cluster Normal 1 EBernhardson Done
#121254 RelatedArticles API query can take 2 seconds High 1 dcausse Done
#121056 Decide on proper error handling for the transferToES.py job Normal 1 - Backlog
#121055 Deploy oozie jobs to calculate popularity score and ship it to elasticsearch Normal 1 - Backlog
#120843 Add cron job to puppet to rebuild completion suggeter indices on a weekly basis. Normal 1 EBernhardson In progress
#120796 Integrate popularity score into completion suggester and full text search scoring Normal 1 dcausse In progress
#120421 Add a UUID to the user testing logs Normal 1 EBernhardson Done
#120281 Make elasticsearch cluster accessible from analytics hadoop workers Normal 1 akosiaris Backlog
#120020 Write A/B test documentation for accept-language header test Normal 1 EBernhardson Done
#119989 Turn on completion suggester beta feature on all Wikimedia wikis on Thursday 17th December 2015 Normal 1 dcausse Done
#119889 Write Completion Suggester Beta Feature user-facing documentation Normal 1 dcausse Done
#119529 Turn off A/B test for the accept-language test on or after 2015-12-17 Normal 1 EBernhardson Done
#119528 Turn on A/B test for the accept-language test on 2015-12-10 after train deployment Normal 1 EBernhardson Done
#119527 Write A/B test for accept-language header, where the user is shown results from other wikis if they get nothing on their primary wiki Normal 1 EBernhardson Done
#118627 generate mobile links in interwiki search results if user is on mobile site High 1 EBernhardson Resolved
#118577 Record if language detection occured in the CirrusSearchUserTesting and CirrusSearchRequestSet logs Normal 1 EBernhardson Done
#118571 Update CirrusSearch-common.php in operations/mediawiki-config to include all languages we can detect well Normal 1 dcausse Done
#118570 Add a map<string,string> field to CirrusSearchRequestSet Normal 1 dcausse Done
#118292 Turn off A/B test for the "fewer than 3 results" test on or after 2015-11-26 Normal 1 EBernhardson Done
#118290 Turn on A/B test for the "fewer than 3 results" test on or after 2015-11-19 Normal 1 EBernhardson Done
#118289 Based on running the tests through the relevance lab, pick a specific one to invest further in Normal 1 Deskana Resolved
#118288 Run test with accept-language header for language through the relevance lab, to decide how promising it is to invest further Normal 1 EBernhardson Resolved
#118287 Run test with different library for detection language through the relevance lab, to decide how promising it is to invest further Normal 1 TJones Done
#118286 Write A/B test for language switching, where the user is shown results from both wikis if they had fewer than 3 from the first Normal 1 Smalyshev Done
#118285 Add instrumentation for measuring clickthroughs when the user is shown results from multiple wikis simultaneously Normal 1 Deskana Resolved
#118197 Turn off language switching test on or after 2015-11-11 Normal 1 Deskana Resolved
#118155 Decide how to deploy avro schemas in hive (CirrusSearchRequestSet) Needs Triage 1 - Resolved
#118053 Reduce logging levels on hypothesis-testing elasticsearch cluster Normal 1 dcausse Done
#117902 Neither search satisfaction schema is producing data despite the fixes Unbreak Now! 1 Ironholds Resolved
#117714 Load test the codfw elasticsearch cluster to verify it can handle production load in a switchover Normal 1 - Backlog
#117575 Setup oozie task for adding and removing CirrusSearchRequestSet partitions in hive Needs Triage 1 dcausse Resolved
#117572 Determine which parts of the language brainstorming session can be run through the relevancy lab Needs Triage 1 EBernhardson Resolved
#117442 Change search satisfaction schema sub test from common terms to language search Needs Triage 1 EBernhardson Resolved
#117181 Elasticsearch index indexing slow log generates too much data High 1 Deskana Resolved
#116872 Comparison Tool for Relevance Lab High 1 Smalyshev Resolved
#116871 Metrics/Report tool for Relevance Lab High 1 TJones Resolved
#116870 JSON Diff tool for Relevance Lab High 1 TJones Resolved
#116869 Enhance runSearch.php to output info needed for Relevance Lab High 1 EBernhardson Resolved
#116576 Cirrus jobs failures: Received job sendData for unwritable cluster 0 0s after insertion Needs Triage 1 EBernhardson Resolved
#116319 Fix and re-deploy satisfaction schema. Unbreak Now! 1 JGirault Done
#116196 TestSearchSatisfaction2 validation errors Needs Triage 1 EBernhardson Resolved
#116137 Investigate loadEventEnd / domComplete regression as of 2015-10-13 Unbreak Now! 1 hashar Resolved
#116055 Build out hadoop job to calculate average page views over time for cirrussearch scoring purposes Normal 1 EBernhardson Done
#116016 Adjust rescoring methods to be able to optionally use additional fields for scoring information Normal 1 dcausse Resolved
#115715 Update CirrusSearchRequestSet schema to have a timestamp field High 1 EBernhardson Resolved
#115626 Set up staging area (beta) running the new portal repository Normal 1 Smalyshev Resolved
#115615 High level design for Relevance Lab High 1 TJones Resolved
#115546 ext.wikimediaEvents throws JS exception on mobile High 1 JGirault Done
#115434 Put together multicluster configuration for mediawiki-config Normal 1 EBernhardson Resolved
#115402 Reindex namespaces after October 22, 2015 Normal 1 EBernhardson Resolved
#115031 Fix the deployed version of the Search schema Unbreak Now! 1 mpopov Resolved
#115024 Code A/B test for AND operator relaxation test Normal 1 EBernhardson Resolved
#114778 Turn A/B test off for AND operator relaxation on 2015-10-20 Normal 1 EBernhardson Resolved
#114774 Turn A/B test on for AND operator relaxation on 2015-10-13 Normal 1 Deskana Resolved
#114773 [EPIC] Run A/B test for AND operator relaxation on 2015-10-13 Normal 1 Deskana Resolved
#113518 trailing \n appearing at the bottom of search results pages on mediawiki.org and meta.wikimedia.org Normal 1 EBernhardson Resolved
#113517 Search test failed Needs Triage 1 EBernhardson Resolved
#113440 build out infra to be able to take data calculated in hadoop and import it to the elasticsearch cluster on a weekly basis Normal 1 Smalyshev Done
#113437 Make the CirrusSearch index updaters take advantage of "adding fields for free" Normal 1 dcausse Resolved
#113291 Add new fields to searchsatisfaction2 schema High 1 EBernhardson Resolved
#113098 Create possibility of retrieving the name of certain Wikimedia project in any language Needs Triage 1 EBernhardson Resolved
#113018 maintenance script to copy the ES index from one cluster to another Normal 1 Smalyshev Resolved
#112863 Cleanup old search indices in eqiad Normal 1 dcausse Resolved
#112681 Analyze how wrong using out of date page view data would be Normal 1 TJones In progress
#112585 Fix CompletionSuggestion data collection and re-start the test. High 1 EBernhardson Resolved
#112504 Install extra plugin 1.7.1 to beta Needs Triage 1 dcausse Resolved
#112499 Release elasticsearch extra plugin 1.7.1 Needs Triage 1 dcausse Resolved
#112349 Inform users what happened if their query is forwarded to another wiki because of language detection Normal 1 JGirault Resolved
#112295 Design and agree on an Avro schema for cirrus search request logging to hadoop High 1 Ironholds Resolved
#112178 Relax 'AND' operator with the common term query High 1 dcausse Resolved
#112169 Put survey on Wikipedia.org page to understand how people use the page Normal 1 TJones Resolved
#112121 Make a demonstration of the interwiki language detection search work in Labs High 1 Smalyshev Resolved
#112028 Implement completion suggester as a Beta Feature Normal 1 dcausse Done
#111694 Bypass query length limit for incategory search Normal 1 Deskana Resolved
#111471 Drop the `phabricatormain` index in production Normal 1 - Resolved
#111293 insource cannot find repeating words Normal 1 Deskana Resolved
#111244 Avoid querying namespace for morelike (optimization) Normal 1 dcausse Resolved
#111137 Override core suggester in AB test between current suggestions and the experimental cirrus-suggest api Needs Triage 1 EBernhardson Resolved
#111120 Run load tests with the completion suggester in prod Normal 1 dcausse Resolved
#111104 Update ElasticSearch documentation to reflect current operations cluster and index building Normal 1 EBernhardson Resolved
#111091 Allow extensions to change the method used to get suggestion results Normal 1 EBernhardson Resolved
#111078 Run A/B test on the search suggester to measure zero results rate, starting on 2015-09-08 High 1 EBernhardson Resolved
#110978 Investigate "no enabled connection" errors being logged recently Needs Triage 1 - Resolved
#110940 Setup auto-update on search-datavis.eqiad.wmflabs Normal 1 EBernhardson Resolved
#110922 Build index for the suggester API for enwiki and dewiki High 1 EBernhardson Resolved
#110915 Improve completion suggester precision High 1 dcausse Resolved
#110883 Add a levenshtein distance score function to the wikimedia extra plugin Normal 1 dcausse Resolved
#110648 [Bug] high-ranking items seemed to have dropped significantly in Special:Search results for wikidata High 1 dcausse Needs review
#110359 Track clicking on suggestion in "Did you mean" results High 1 TJones Resolved
#110231 Document the resolved redirect fix so consumers can update their code Normal 1 EBernhardson Resolved
#110226 Produce follow-up analysis of zero results queries to see what our impact has been and how patterns have changed High 1 TJones Resolved
#110179 Update Elasticsearch for missing updates from outage on 20150825 Normal 1 EBernhardson Resolved
#110078 Query elasticsearch language detection plugin and then issue query to suggested language wiki(s) Normal 1 Smalyshev Resolved
#110077 Install language detection plugin to vagrant for testing Needs Triage 1 dcausse Resolved
#110075 Make it possible to load the CirrusSearch configuration of another wiki in production Normal 1 Smalyshev Resolved
#109734 enable cirrussearch to talk to two clusters Needs Triage 1 EBernhardson Resolved
#109731 Run tests to measure the expected zero results change of running zero result enwiki queries against other languages (ru, jp, etc?) Normal 1 TJones Resolved
#109729 Run tests to measure the expected zero results change of changing suggestions algorithm Needs Triage 1 EBernhardson Resolved
#109482 Implement User Satisfaction Test Schema 2.0.0 Needs Triage 1 EBernhardson Resolved
#109151 Run tests to measure the expected zero results change of adjusting phrase slop Needs Triage 1 TJones Resolved
#109122 CirrusSearch should send instances of Search backend error to graphite Normal 1 EBernhardson Resolved
#109018 Run an A/B test based on the phrase slop setting in Cirrus High 1 EBernhardson Resolved
#108869 Switch A/B test logs over to a more easily analysable format High 1 EBernhardson Resolved
#108554 index appears to be missing from search API responses if the page is a resolved redirect High 1 dcausse Resolved
#108103 Turn on A/B test varying confidence threshold and smoothing Normal 1 Deskana Resolved
#108043 Full review of small sample (~1K) of full text queries to categorize them all High 1 TJones Resolved
#108034 Fix a bug in PhraseSuggester that prevents us to use the reverse field Needs Triage 1 dcausse Resolved
#107947 Set a hard byte or character limit for queries Normal 1 TJones Resolved
#107907 Add additional fields to A/B testing logs in Cirrus Normal 1 EBernhardson Resolved
#107778 Increase the sample rate for the TestUserSearchSatisfaction Schema Needs Triage 1 EBernhardson Resolved
#107666 Exact phrase match trumps title match Low 1 TJones Resolved
#107413 cirrussearch-didyoumean-settings includes confusing sentences Normal 1 dcausse Resolved
#107411 Upgrade beta to elasticsearch 1.7.1 Needs Triage 1 dcausse Resolved
#107348 Timeouts when trying to create mappings. Normal 1 EBernhardson Done
#107278 Use fixed ports for elasticsearch Needs Triage 1 Dzahn Resolved
#107171 Create connector porting the A/B test Cirrus logfiles to stat1002 from Fluorine Normal 1 EBernhardson Resolved
#107164 Run tests varying the options to the suggestions api Needs Triage 1 Deskana Resolved
#107163 Decide on sets of suggestion api variations to try in production user testing Needs Triage 1 Deskana Resolved
#107088 Investigate why some suboptimal suggestions are provided Normal 1 dcausse Resolved
#107035 Review elastic search logs for useful patterns of search activity that get no results. High 1 TJones Resolved
#106888 Build backend A/B test infrastructure to measure zero result rate of different search ideas Needs Triage 1 EBernhardson Resolved
#106692 Allow customization of "Did you mean" suggestions. Needs Triage 1 dcausse Resolved
#106691 Write a script that send queries to the production cluster and collect the results Needs Triage 1 EBernhardson Resolved
#106681 The morelike special syntax should be considered as a special syntax Normal 1 dcausse Resolved
#106552 Add some sort of user identifier to CirrusSearchRequests log to collapse multiple prefix searches from the same user within a short time period for measuring zero results purposes Normal 1 EBernhardson Resolved
#106391 Speed up regex matching Low 1 Manybubbles Resolved
#106336 Long unicode queries cause hhvm warning: "determinizing automaton would result in more than 10000 states" High 1 EBernhardson Resolved
#106165 Upgrade production to elasticsearch 1.7.1 Normal 1 chasemp Resolved
#106164 Upgrade beta to Elasticsearch 1.7.0 Normal 1 EBernhardson Resolved
#106160 Validate Cirrus against Elasticsearch 1.7.0 Normal 1 Manybubbles Resolved
#106132 Update elasticsearch swift repository plugin Needs Triage 1 Manybubbles Resolved
#106129 Write the API that aggregates multiple suggestion and returns ranked suggestions to the client Needs Triage 1 Smalyshev Resolved
#106128 Write a preliminary scoring function to rank suggestions Normal 1 dcausse Resolved
#106127 Write a script that creates the completion suggester index Normal 1 dcausse Resolved
#105746 Build out an API that exposes ElasticSearch suggester results for a given query Normal 1 Smalyshev Resolved
#105204 Get more data on how many users are getting "Did you mean" suggestions Normal 1 - Resolved
#105202 If the user gets zero results, but gets a "Did you mean" result, just run the query for the "Did you mean" result and inform the user that this happened Normal 1 EBernhardson Resolved
#105201 Measure whether users used the "did you mean" results that we serve to users somehow Normal 1 EBernhardson Resolved
#105184 Parallelize the theory-testing pipeline Normal 1 Deskana Resolved
#105183 Speed up the theory-testing pipeline High 1 Deskana Resolved
#105182 Ensure that zero-results rate progresses according to plan High 1 EBernhardson Resolved
#105181 Monitor the progress of the zero-results reduction Normal 1 - Resolved
#105180 Establish a Q1 target progression for the reduction of zero-results Normal 1 Deskana Resolved
#104908 PHP Notice: Undefined property: Message::$isDisabled Unbreak Now! 1 Manybubbles Resolved
#104560 Upgrade Elastica extension Low 1 Manybubbles Resolved
#103872 Grant access to HTTP request logs Normal 1 RobH Resolved
#103598 Upgrade Beta to Elasticsearch 1.6.0 High 1 dcausse Resolved
#103597 Release Elasticsearch plugins for 1.6.0 to archiva Normal 1 dcausse Resolved
#103505 Create analytics-centric Cirrus logs and have them import into HDFS High 1 EBernhardson Resolved
#103000 Package the discovery teams dashboards Needs Triage 1 EBernhardson Resolved
#102895 Add "Give us your feedback" link to bottom of search page on English Wikipedia on mobile, as well Needs Triage 1 EBernhardson Resolved
#102864 SWAT deploy configuration change to turn on search feedback form on English Wikipedia Normal 1 Jdouglas Resolved
#102594 Figure out why Elasticsearch doesn't recover from out of memory - even after we bounce the node that was out of memory High 1 Manybubbles Resolved
#102429 Cirrus logs on stat1002 broken(?) Unbreak Now! 1 Manybubbles Resolved
#102279 Clean Elasticsearch log in mediawiki vagrant High 1 EBernhardson Resolved
#102277 Figure out why some queries the produce zero results have "{searchTerms}" as the search term Normal 1 EBernhardson Resolved
#102042 Log whether or not the request returned a suggestion in the CirrusSearchRequests.log Normal 1 - Resolved
#102008 Import Elasticsearch 1.6.0 deb into wmf apt Normal 1 Manybubbles Resolved
#101840 CirrusSearch: insource/incategory-search give no results when combined with multiple spaces in between Normal 1 dcausse Resolved
#101783 Add "Give us your feedback" link to bottom of search page on English Wikipedia Needs Triage 1 Jdouglas Resolved
#101691 Dump Cirrus index into a file Normal 1 dcausse Resolved
#101111 Allow customization of morelike for experimentation Needs Triage 1 dcausse Resolved
#100907 Implement an EventLogging schema for session tracking Normal 1 EBernhardson Resolved
#100838 JobRunner database connection journey Normal 1 - Resolved
#100791 The cirrus links update job tests no longer work High 1 - Resolved
#100500 Install elasticsearch-statsd-plugin to the beta cluster Needs Triage 1 EBernhardson Resolved
#100468 Enumerate the existing searchey API capabilities Normal 1 EBernhardson Resolved
#99794 Insert # of docs, # of deleted docs and ratio of deleted docs for select indicies to ganglia Needs Triage 1 EBernhardson Resolved
#99005 es-tool restart-fast can timeout Normal 1 fgiunchedi Resolved
#98786 CirrusSearch: Stop logging an error when I user sends a prefix query that is too long Normal 1 EBernhardson Resolved
#98190 CirrusSearch: More data in CirrusSearchRequests logs High 1 EBernhardson Resolved
#97854 CirrusSearch: Release wikimedia-extra and experimental-highlighter for Elasticsearch 1.5 Normal 1 Manybubbles Resolved
#97408 Release wikimedia-extra plugin to archiva and beta High 1 Jdouglas Resolved
#97310 click-result events underreported in desktop search event logging High 1 bmansurov Resolved
#97173 Deploy new version of experimental highlighter and wikimedia extra to prod with an Elasticsearch rolling restart High 1 - Resolved
#97172 Release new version of experimental highlighter to wikimedia archiva and deploy to beta High 1 Jdouglas Resolved
#92796 Prefix search API doesn't return "index" field when "redirects" is enabled. High 1 EBernhardson Resolved
#91666 CirrusSearch: * or ? at the start of a word is ignored Normal 1 EBernhardson Resolved
#89823 CirrusSearch: Allow *ORs* of incategory to be sent via a post or get parameter Normal 1 Manybubbles Resolved
#88247 insource should search article text on non-wikitext pages. Probably. Low 1 ori Done
#86781 Support multiple datacenters in CirrusSearch Normal 1 EBernhardson Resolved
#77928 Remove usage of Query->setFilter from Cirrus Normal 1 - Resolved
#76497 Add per user concurrent search request limiting Normal 1 EBernhardson Resolved
#75374 Parse errors with extra (ideographic) spaces in query High 1 EBernhardson Resolved