OPTIMAL STOPPING METHODOLOGY FOR THE SECRETARY PROBLEM WITH RANDOM QUERIES BY GEORGE V. MOUSTAKIDES1,XUJUN LIU2,* AND OLGICA MILENKOVIC2,† 1Department of ECE, University of Patras, 26500 Rion, GREECE. [email protected] 2Department of ECE, University of Illinois, Urbana-Champaign, IL, USA. *[email protected]; †[email protected] Candidates arrive sequentially for an interview process which results in them being ranked relative to their predecessors. Based on the ranks available at each time, one must develop a decision mechanism that selects or dismisses the current candidate in an effort to maximize the chance to select the best. This classical version of the “Secretary problem” has been studied in depth using mostly combinatorial approaches, along with numerous other variants. In this work we consider a particular new version where during reviewing one is allowed to query an external expert to improve the probability of making the correct decision. Unlike existing formulations, we consider experts that are not necessarily infallible and may provide suggestions that can be faulty. For the solution of our problem we adopt a probabilistic methodology and view the querying times as consecutive stopping times which we optimize with the help of optimal stopping theory. For each querying time we must also design a mechanism to decide whether we should terminate the search at the querying time or not. This decision is straightforward under the usual assumption of infallible experts but, when experts are faulty, it has a far more intricate structure. 1. Introduction. The secretary problem, also known as the game of Googol, the beauty contest problem, and the dowry problem, was formally introduced in [6] while the first so- lution was obtained in [10]. A widely used version of the secretary problem can be stated as follows: n individuals are ordered without ties according to their qualifications. They apply for a “secretary” position, and are interviewed one by one, in a random order. When the t- th candidate appears, s/he can only be ranked (again without ties) with respect to the t ´ 1 already interviewed individuals. At the time of the t-th interview, the employer must make the decision to hire the person present or continue with the interview process by rejecting the candidate; rejected candidates cannot be revisited at a later time. The employer succeeds only if the best candidate is hired. If only one selection is to be made, the question of interest is to determine a strategy (i.e., final rule) that maximizes the probability of selecting the best candidate. arXiv:2107.07513v2 [cs.DS] 4 Aug 2021 In [10] the problem was solved using algebraic methods with backwards recursions while [3] considered the process as a Markov chain. An extension of the secretary problem, known as the dowry problem with multiple choices (for simplicity, we refer to it as the dowry prob- lem), was studied in [7]. In the dowry problem, one is allowed to select s ¡ 1 candidates during the interview process, and the condition for success is that the selected group includes the best candidate. This review process can be motivated in many different ways: for ex- ample, one may view the s-selection to represent candidates invited for a second round of interviews. In [7] we find a heuristic solution to the dowry problem while [17] solved the problem using a functional-equation approach of the dynamic programming method. MSC2020 subject classifications: Primary 60G40, Stopping Times, Optimal Stopping; secondary 62L15, Op- timal Stopping in Statistics. Keywords and phrases: Multiple stopping times, Secretary problem, Dowry problem. 1 2 In [7] the authors also offer various extensions to the secretary problem in many different directions. For example, they examined the secretary problem (single choice) when the ob- jective is to maximize the probability of selecting the best or the second-best candidate, while the more generalized version of selecting one of the top ` candidates was considered in [8]. Additionally in [7] the authors studied the full information game where the interviewer is al- lowed to observe the actual values of the candidates, which are chosen independently from a known distribution. Several other extensions have been considered in the literature: the post- doc problem, for which the objective is to find the second-best candidate [7, 16]; the problem of selecting all or one of the top ` candidates when ` choices are allowed [14, 19, 20]. More recently, the problem is considered under a model where interviews are performed according to a nonuniform distribution such as the Mallows or Ewens, [2,9, 11, 12]. For more details regarding the history of the secretary problem, the interested reader may refer to [4,5]. The secretary problem with genie-queries, an extension of the secretary problem with multiple choices, was introduced in [12] and solved using combinatorial methods for a gen- eralized collection of distributions that includes the Mallows distribution. In this extended version it is assumed that the decision making entity has access to a limited number of in- fallible genies (experts). When faced with a candidate in the interviewing process a genie, if queried, provides a binary answer of the form “The best” or “Not the best”. Queries of experts are frequently used in interviews as additional source of information and the feed- back is usually valuable but it does not mean that it is necessarily accurate. This motivates the investigation of the secretary problem when the response of the genie is not deterministic (infallible) but it may also be false. This can be modeled by assuming that the response of the genie is random. Actually, the response does not even have to be binary as in the random query model employed in [1, 13] for the completely different problem of clustering. In our analysis we consider more than two possibilities which could reflect the level of confidence of the genie in its knowledge about the sample being the best or not. For example a four-level response could be of the form: “The best with high confidence”, “The best with low confi- dence”, “Not the best with low confidence” and “Not the best with high confidence”. As we will see, the analysis of the problem under a randomized genie model requires stochastic op- timization techniques and in particular results we are going to borrow from optimal stopping theory [15, 18]. 2. Background. We begin by formally introducing the problem of interest along with our notation. Suppose the set tξ1; : : : ; ξnu contains samples that are random uniform draws n without replacement from the set of integers t1; : : : ; nu. The sequence tξtut“1 becomes avail- able sequentially and we are interested in identifying the sample with value equal to 1, which is regarded as “the best”. The difficulty of the problem stems from the fact that the value ξt is not observable. Instead, at each time t, we observe the relative rank zt of the sample ξt after it is being compared to all the previous samples tξ1; : : : ; ξt´1u. If zt “ m (where 1 ¤ m ¤ t) this signifies that in the set tξ1; : : : ; ξt´1u there are m ´ 1 samples with values strictly smaller than ξt. As mentioned, at each time t we are interested in deciding whether tξt “ 1u or tξt ¡ 1u based on the information provided by the relative ranks tz1; : : : ; ztu. Consider now the existence of an expert/genie which we may query. The purpose of query- ing at any time t is to obtain from the genie the information about the current sample being the best or not. Unlike all articles in the literature that treat the case of deterministic genie responses here, as mentioned in the Introduction, we adopt a random response model. In the deterministic case the genie provides the exact answer to the question of interest and we ob- viously terminate the search if the answer is “tξt “ 1u”. In our approach the corresponding response is assumed to be a random number ζt that can take M different values. The reason we allow more than two values is to model the possibility of different levels of confidence in THE SECRETARY PROBLEM WITH RANDOM QUERIES 3 the genie response. Without loss of generality we may assume that ζt P t1;:::;Mu and the probabilistic model we adopt is the following (1) Ppζt “ m|ξt “ 1q “ ppmq; Ppζt “ m|ξt ¡ 1q “ qpmq; m “ 1; : : : ; M; M M where m“1 ppmq “ m“1 qpmq “ 1, to assure that the genie responds with probability 1. As we can see, the probability of the genie generating a specific response depends on whether ř ř the true sample value is 1 or not. Additionally, we assume that ζt conditioned on tξt “ 1u or tξt ¡ 1u is statistically independent from any other past or future responses, relative ranks and sample values. It is clear that the random model is more general than its deterministic counterpart. Indeed we can emulate the deterministic case by simply selecting M “ 2 and pp1q “ 1; pp2q “ 0; qp1q “ 0; qp2q “ 1, with ζt “ 1 corresponding to “tξt “ 1u” and ζt “ 2 to “tξt ¡ 1u” with probability 1. In the case of deterministic responses it is clear that we gain no extra information by querying the genie more than once per sample (the genie simply repeats the same response). Motivated by this observation we adopt the same principle for the random response model as well, namely, we allow at most one query per sample. Of course, we must point out that under the random response model, querying multiple times for the same sample makes perfect sense since the corresponding responses may be different. However, as mentioned, we do not adopt this possibility in our current analysis.