Microsoft Research shows that logarithmic mapping allows for low discount factors by creating action gaps similar in size

While reinforcement learning (RL) has seen significant successes over the past few years, modern deep RL methods are often criticized for how sensitive they are with respect to their hyper-parameters. One such hyper-parameter is the discount factor, which controls how future rewards are weighted compared to immediate rewards.

The objective that one wants to optimize in RL is often best described as an undiscounted sum of rewards (for example, maximizing the total score in a game). 

In practice, however, a discount factor is introduced to avoid some of the optimization challenges that can occur when directly optimizing on an undiscounted objective. And while in theory a discount factor can take on any value between 0 and 1, in reality good performance is only obtained for a small subset of values close to 1.

Microsoft Research will present a paper at the thirty-third Conference on Neural Information Processing Systems (NeurIPS 2019), called “Using a Logarithmic Mapping to Enable Lower Discount Factors in Reinforcement Learning.” 

The company will present a novel hypothesis as to why the effective discount-factor range is so small. Also, based on its hypothesis, Microsoft outlined a technique to avoid this discount-factor sensitivity and introduce a method, which is called logarithmic Q-learning, based on this technique. 

Logarithmic Q-learning is the first method able to achieve good performance for low discount factors on sparse-reward tasks with function approximation. In addition, the method not only reduces discount-factor sensitivity, but can also improve performance altogether.

While a logarithmic mapping function makes action gaps more homogenous, there are a number of challenges to overcome to build a robust algorithm, based on this outcome, that can be applied to stochastic domains with both positive and negative rewards. 

Microsoft managed to overcome these challenges and develop an algorithm, logarithmic Q-learning, that can be applied to general tasks and has convergence guarantees under standard conditions. These results do not only show great performance for low discount factors. Early performance of high discount factors is also better than it was without the logarithmic mapping.

Kaspersky set to process and store customer data from the United States and Canada in Switzerland

Kaspersky is progressing with the relocation of its data storage and processing infrastructure to Switzerland by moving data from customers from the United States and Canada. In addition, the company is announcing plans to open its first Latin American Transparency Center in São Paulo, Brazil in January 2020.

As a global cybersecurity company, Kaspersky has been committed to the strongest protection of its customers’ data. Moreover, the company has been working to improve customer security by building effective data protection based on digital trust. Therefore, as part of its Global Transparency Initiative, Kaspersky has been adapting its data storage and processing infrastructure by moving core processes from Russia to Switzerland – starting with European users.

The company is now taking the next step by announcing that data from customers of the United States and Canada will also be moved and processed in Switzerland. This data is shared voluntarily with the Kaspersky Security Network (KSN) – an advanced, cloud-based system that automatically processes cyberthreat-related data. This includes suspicious or previously unknown malicious files that the company’s products send to KSN for automated malware analysis.

In addition, Kaspersky has announced plans to open a new Transparency Center in Latin America. It’s been exactly one year since the company established its first Transparency Center in Zurich – a dedicated security facility for trusted partners to review the company’s source code – and it proved to be an important and necessary tool for both existing and prospective Kaspersky customers to increase their confidence and trust in cybersecurity products and services.

Delivering on customers’ needs, earlier this year the company opened one more European Transparency Center in Madrid and announced the opening of its APAC Transparency Center in Malaysia, in addition to adding extended functionalities to the facilities. Kaspersky provides its visitors with the possibility to learn more about engineering and data-processing practices, as well as its product portfolio. 

“All of us want to protect our own data,” said Anton Shingarev, vice president for public affairs at Kaspersky. “That is an understandable demand in our digital world. But we need to know that we can trust our own choices in digital technology, like smartphones, laptops, mobile applications or cybersecurity solutions. Trust needs to be earned, and at Kaspersky we believe any company needs to constantly improve transparency and accountability in cyberspace. We are glad to proceed further with our big infrastructure project and, as promised, moving other regions in addition to European customers, whose data is processed in Swiss data centers."

At all of Kaspersky’s Transparency Centers, the company provides the opportunity to compile the company’s software from its source code and compare it with those that are publicly available. This move assures an unprecedented level of confidence in Kaspersky’s products, allowing them to run a compilation process with the assistance of the company’s experts.

“Our specifically developed ‘three-layer’ approach to security briefings has proved successful since it provides each partner with the most appropriate and needed type of support,” Shingarev added. “From sharing our transparency and data processing practices, to conducting a hardcore comprehensive source code review, we provide our partners with the information they need. No single cybersecurity vendor has gone as far in terms of transparency over its processes and technologies, and we are happy to take a lead to make the entire industry more open and reliable.”

Also, Kaspersky has been continuously working on its Bug Bounty Program. Since March 2018, the company has received and resolved 66 bugs. In total, 20 reports were rewarded with combined bounties totaling US$45,350.

Data Dimensions introduces PANOPTIC, its agnostic, end-to-end medical bills processing platform

Data Dimensions launched Thursday PANOPTIC, its complete processing platform built for the auto casualty and work compensation markets.

The PANOPTIC platform is completely agnostic, integrating with any bill review company, making it a versatile asset to insurers. With features such as provider credentialing, fraud and abuse detection, bill screening, filtering and routing, PANOPTIC is a comprehensive cost containment tool. The platform can also identify and help to prevent provider network leakage.

The PANOPTIC platform has the ability to receive claims documentation, medical bills and their attachments in any format (Paper mail, Fax, email, eBill, direct upload); an Intelligent Business Rules Engine applies ICD-9/ICD-10 validation, medical bills filtering, provider screening and claimant eligibility before routing the bills to the appropriate network, payor or bill review platform.

In addition, PANOPTIC helps insurers to meet Insurance Diversity Initiative (IDI) goals, pay providers and non-providers electronically, and provides actionable data through analytics and reporting. 

Artificial Intelligence is applied throughout the platform for maximum accuracy and efficiency, and its proprietary tracking and monitoring component gives visibility at every stage of the process.