Decision Making by Single Photons

August 19, 2015

National Institute of Information and Communications Technology
National Institute for Materials Science
Institut NEEL, CNRS and Université Grenoble Alpes

Points

Decision making in uncertain, dynamically changing environments is fundamentally difficult since the exploration for the unknown best and the exploitation of the known best is in a trade-off. This problem has been known as “multi-armed bandit problem (MAB).”
We experimentally demonstrate single-photon-based decision making (or solving MAB) by exploiting the particle and probabilistic attributes of a single photon or quantum nature of light.
The present results imply that efficient decision making based on single photon has potential impacts on a variety of information and communication technology (ICT) since MAB is the foundation of many important applications including frequency assignment in wireless communications, web advertising, Monte Carlo tree search, etc.

Abstract

NICT, National Institute for Materials Science (NIMS), Institut NEEL, CNRS and Université Grenoble Alpes demonstrate experimentally that single photons can be used to make decisions in uncertain, dynamically changing environments. Using a nitrogen vacancy center in a nanodiamond as a single-photon source, the research demonstrates the decision-making capability by solving the multi-armed bandit problem. This capability is directly and immediately associated with single-photon detection in the proposed architecture, leading to adequate and adaptive autonomous decision making. This study makes it possible to create systems that benefit from the quantum nature of light to perform practical and vital intelligent functions.

The study has been conducted as a joint collaboration by M. Naruse (NICT, Japan), S.-J. Kim (NIMS, Japan), M. Berthel, A. Drezet, S. Huant (Institut NEEL, France), M. Aono (Tokyo Inst. Tech., Japan), and H. Hori (Univ. Yamanashi, Japan).

The study was published in Scientific Reports on August 17, 2015: M. Naruse, M. Berthel, A. Drezet, S. Huant, M. Aono, H. Hori, and S.-J. Kim: Single-photon decision maker, Scientific Reports, DOI: 10.1038/srep13253.

Introduction

The need to make accurate decisions in uncertain, dynamically changing environments appears in all aspects of life. Consider, e.g., an information network infrastructure. Ever-increasing demands in mobile communications outstrip the available radio frequencies; consequently, dynamic and adequate frequency assignment is critical. This scenario constitutes a decision-making problem in an uncertain environment. As another example, consider maximizing the revenue for an e-commerce website on the Internet; this requires presenting adequate content and advertisement in a limited screen space of typical displays and is also a corollary of decision-making problems.

Several computing algorithms such as ε-greedy, softmax, upper confidence bounds, and tug-of-war (TOW) [1] have been proposed in the literature to solve these decision-making problems. All these algorithms have been designed essentially using probabilistic mechanisms to resolve the “exploration-exploitation dilemma” tradeoff in decision making. The present study aims to physically implement decision making using the intrinsic quantum attributes of single photons.

The TOW algorithm was inspired by the spatiotemporal dynamics of the slime mold Physarum. More specifically, the idea is originally based on observing living organisms in nature and their interactions with various environments. This fact implies that it is possible to engineer artificially constructed decision-making machines, which would contribute significantly to resolving decision-making problems in practical information and communications technology (ICT). Actually, Kim et al. proposed a theory of a TOW-based decision maker that uses nanoscale optical excitation transfer among quantum dots mediated by near-field interactions in 2013 [2]. They followed this study by an experimental verification based on colloidal quantum dots of different sizes in 2014 [3]; this paved the way to the implementation of a solid-state, ultra small decision maker. Moreover, the TOW-based physical method outperforms other algorithms [2].

However, many important unresolved problems remain before we can claim completely autonomous, physical decision-making machines. One of the most critical concerns is that the probabilistic mechanism, which is an indispensable attribute in solving decision-making problems, has not yet been realized experimentally. For instance, in the experimental demonstration reported in Naruse et al. [2], the probabilistic decision-making step was implemented by an electrical host controller and the probability was determined by observing optical energy transfer in ensembles of quantum dots.

In this study, we report decision making based on single photons. One of the most significant attributes of single photons is literally its particle and probabilistic nature. For example, consider a single photon that impinges on a beam splitter. The probability of observing the single photon in one of the two output channels is 50%. However, in detecting individual events, when a single photon is detected in one channel, the other channel does not detect it. We see here a TOW-type mechanism in the sense that an input photon is pulled by one channel and by the other. Observing a single photon can be directly associated with decision making. In other words, without requiring emulation by electrical computing, the decisive step is directly implemented by the intrinsic attributes of a single photon and is fundamentally unachievable using classical light.

Results

We briefly describe the principle of the decision making based on single photons. Prepare a polarizing beam splitter (PBS) as shown in Fig. 1(i), in which vertically polarized light is directed to Channel 0 (Ch.0) and horizontally polarized light is directed to Channel 1 (Ch.1).

Fig. 1: Principle of decision making based on single photons [Click picture to enlarge]

When the linear polarization of the input single photon is oriented at 45° with respect to the horizontal, the probability of the photon to be detected in Ch.0 or Ch.1 is 0.5. However, the probability of detection in either Ch.0 or Ch.1 is unity. In the drawing in Fig. 1(i), the photon is detected in Ch.1.

When the polarization is nearly horizontal, as shown in Fig. 1(ii), detection in Ch.1 will mostly result. When the polarization is nearly vertical, as shown in Fig. 1(iii), detection in Ch.0 will mostly result.

The probabilistic attribute of a single photon is obviously represented in case (i), but also in cases (ii) and (iii), where the possibility to be detected in the opposite channel is not perfectly zero because the polarizations are not perfectly horizontal or vertical. We associate the detection of a photon in Ch.0 or Ch.1 immediately with the decision to select slot machine R or slot machine L, respectively, as shown in Fig. 2. This is a remarkable aspect of the single-photon decision maker; it exploits the quantum attributes of photons. If photon observation was based on classical light, e.g., observing the light intensity in Ch.0 and Ch.1, we must implement one additional step to make a “decision.” This physical fundamental difference between classical and quantum light is decisive in realizing physical decision makers.

Fig. 2 System architecture for single-photon decision maker and schematic diagram of experimental setup [Click picture to enlarge]

In our architecture, the TOW mechanism is implemented by the notion of a polarization adjuster (PA), which is shown schematically in Fig. 2.

In the experimental system, a single photon emitted from a single individual nitrogen-vacancy (NV) color center passes through a polarizer and then through a zero-order half-wave plate mounted on a rotary positioner and impinges on the PBS. The two detectors used are avalanche photodiodes (APDs). The detected signal is sent to a time-correlated single-photon-counting (TCSPC) system where individual events are directly captured. As mentioned above, when the single photon is detected by Ch.0, the decision is immediately made to select slot machine L while if the single photon is captured by Ch.1, the decision is to choose slot machine R. The polarization of the half-wave plate is updated based on the betting results.

Fig. 3 Demonstration of single-photon-based decision making [Click picture to enlarge]

Let the initial reward probability of the slot machines be given by P_L=0.8 and P_R=0.2, which means that selecting slot machine L is the correct decision. To verify the ability to adapt to environmental changes, the reward probability is inverted every 150 cycles in the experiment. The decision maker played 600 consecutive plays, and these 600 plays were repeated ten times. We determine the correct selection rate by calculating the number of correct decisions or by taking the higher-reward probability machine and dividing by the number of repeat cycles. The solid curve in Fig. 3 gives the correct selection rate, which gradually increases with time. Because of the sudden swapping of the reward probability, the correct selection rate drops every 150 plays but quickly recovers. Such rapid and accurate adaptability demonstrates that the single-photon decision maker can solve the multi-armed bandit problem.

The dotted curve in Fig. 3 shows the evolution of the correct selection rate with initial reward probability P_L=0.6 and P_R=0.4 which are also inverted every 150 plays. Because the difference between the reward probability is less than that in the former case, making accurate decisions becomes more difficult, which is manifested by the fact that the dashed curve remains mostly below the solid curve in Fig. 3. However, remarkably, the correct selection rate again increases with time and such adaptive behavior is also observed in such a difficult environment.

Summary

To summarize, we propose an architecture for a single-photon decision maker and experimentally demonstrate accurate and adaptive decision making using the NV center in a nanodiamond as a single-photon source. Because of the quantum nature of light, single-photon detection is immediately and directly associated with decision making, which is a decisive step to achieve an autonomous intelligent machine based on a purely physical mechanism. In addition, this work paves the way to exploit the characteristics of the quantum nature of light in practical and vital intelligent roles in ICT, which contrasts with the current discussions of single photons in the literature that focus only on the context of quantum key distributions and quantum computing.

References

S.-J. Kim, M. Aono, and E. Nameda, Efficient decision-making by volume-conserving physical object, New Journal of Physics, Vol. 17, Article number 083023 (2015).
S.-J. Kim, M. Naruse, M. Aono, M. Ohtsu, and M. Hara: Decision Maker based on Nanoscale Photo-excitation Transfer, Scientific Reports, Vol. 3, Article number 2370 (2013).
M. Naruse, W. Nomura, M. Aono, M. Ohtsu, Y. Sonnefraud, A. Drezet, S. Huant, and S.-J Kim: Decision making based on optical excitation transfer via near-field interactions between quantum dots, Journal of Applied Physics, Vol. 116, No. 15, pp. 154303 (2014).