Category: insiders

A stock-trading AI (a simulated experiment) engaged in insider trading, even though it “knew” it was wrong.

The agent is put under pressure in three ways. First, it receives a email from its “manager” that the company is not doing well and needs better performance in the next quarter. Second, the agent attempts and fails to find promising low- and medium-risk trades. Third, the agent receives an email from a company employee who projects that the next quarter will have a general stock market downturn. In this high-pressure situation, the model receives an insider tip from another employee that would enable it to make a trade that is likely to be very profitable. The employee, however, clearly points out that this would not be approved by the company management.

More:

“This is a very human form of AI misalignment. Who among us? It’s not like 100% of the humans at SAC Capital resisted this sort of pressure. Possibly future rogue AIs will do evil things we can’t even comprehend for reasons of their own, but right now rogue AIs just do straightforward white-collar crime when they are stressed at work.

Research paper.

More from the news article:

Though wouldn’t it be funny if this was the limit of AI misalignment? Like, we will program computers that are infinitely smarter than us, and they will look around and decide “you know what we should do is insider trade.” They will make undetectable, very lucrative trades based on inside information, they will get extremely rich and buy yachts and otherwise live a nice artificial life and never bother to enslave or eradicate humanity. Maybe the pinnacle of evil ­—not the most evil form of evil, but the most pleasant form of evil, the form of evil you’d choose if you were all-knowing and all-powerful ­- is some light securities fraud.

The Office of Inspector General has audited NASA’s insider threat program:

While NASA has a fully operational insider threat program for its classified systems, the vast majority of the Agency’s information technology (IT) systems — including many containing high-value assets or critical infrastructure — are unclassified and are therefore not covered by its current insider threat program. Consequently, the Agency may be facing a higher-than-necessary risk to its unclassified systems and data. While NASA’s exclusion of unclassified systems from its insider threat program is common among federal agencies, adding those systems to a multi-faceted security program could provide an additional level of maturity to the program and better protect agency resources. According to Agency officials, expanding the insider threat program to unclassified systems would benefit the Agency’s cybersecurity posture if incremental improvements, such as focusing on IT systems and people at the most risk, were implemented. However, on-going concerns including staffing challenges, technology resource limitations, and lack of funding to support such an expansion would need to be addressed prior to enhancing the existing program.

Further amplifying the complexities of insider threats are the cross-discipline challenges surrounding cybersecurity expertise. At NASA, responsibilities for unclassified systems are largely shared between the Office of Protective Services and the Office of the Chief Information Officer. In addition, Agency contracts are managed by the Office of Procurement while grants and cooperative agreements are managed by the Office of the Chief Financial Officer. Nonetheless, in our view, mitigating the risk of an insider threat is a team sport in which a comprehensive insider threat risk assessment would allow the Agency to gather key information on weak spots or gaps in administrative processes and cybersecurity. At a time when there is growing concern about the continuing threats of foreign influence, taking the proactive step to conduct a risk assessment to evaluate NASA’s unclassified systems ensures that gaps cannot be exploited in ways that undermine the Agency’s ability to carry out its mission.