Governance and AI: Fiduciary Duties

Fiduciary Duties

Nonprofit board members don’t need to be technologists to meet their fiduciary duties around AI—but they do need to meet their fiduciary duties:

  1. Duty of care
  2. Duty of loyalty

Duty of Care

Board members must act with reasonable care, including by understanding how AI is being used, what risks it creates, and how to manage related issues of compliance and risk management. Reasonable care includes reasonable inquiry or investigation. Boards should not passively wait for management to raise these issues, if that has yet to happen, and instead should provide sufficient support to develop practices, guidance, and oversight over the organization’s use and management of AI.

Duty of Loyalty

Board members must also act in the best interests of the organization in light of its mission and core values. The organization’s best interests may involve using AI in ways that help provide services more effectively, more efficiently, and in a way that serves more people. However, this must be weighed against the adverse effects the AI may have on the organization’s mission, core values, intended beneficiaries, culture, and employee health, and on the environment and ecosystem upon which the organization is dependent. Further, board members should also properly manage any actual or perceived conflicts of interest and keep confidential sensitive board discussions.

What Should the Board and Staff Know?

  • How AI is currently being used by the organization
  • How AI might be used by the organization
  • Whether the organization has capacity or needs to invest in capacity to ensure that its use of AI is compliant, beneficial, in furtherance of its purpose, and does not place the organization or its community at unreasonable risk
  • Whether there are policies and plans in place or that need to be developed to ensure that the organization’s use of AI is beneficial and in furtherance of its purpose (e.g., AI Use Policy, Data Use Policy)
  • Whether AI might be used to help the board in its sense- and decision-making
  • Whether the organization’s use of AI would create any concerns if widely known by the organization’s communities, including beneficiaries, clients, donors, funders, allies, adversaries, and regulators

What Should the Staff Additionally Know?

  • Whether the organization has identified the risks created by its use of AI
    • Purpose- and values-related risks
    • Bias and discrimination
    • Privacy and data security
    • Accuracy and hallucinations
    • Cyber-risks
    • Reputational risks
  • Whether there are ways to reduce risks created by the organization’s use of AI (generally communicated through thoughtfully developed and regularly updated policies, rigorously enforced, backed by credible advisors, and promoted with tone from the top)
    • Permissible AI uses (e.g., background information used general for familiarization, edits of various communications not reliant on legal or professional accuracy)
    • Impermissible AI uses without appropriate human review (e.g., employment decisions; substitute for a legal, tax, or critical professional opinion; communications of great importance, particularly if tone and context are critical elements)
    • Permissible data to feed AI or enter into prompts
    • Impermissible data to feed AI or enter into prompts (e.g., personally identifiable information, attorney-client privileged communications, confidential information and sensitive information if not sufficiently contained and secured)
    • Accuracy checks (e.g., with actual and credible sources and citations)
    • Bias checks (e.g., is guidance or work provided by AI perpetuating biases built-in to the data used by the AI system?)
    • Vendor checks
    • Insurance
    • Incident response and contingency plans (e.g., to address breaches, hacks, misrepresentations, failures, outages, complaints, and other AI-related problems)

What About the Existential and Broad Societal Risks?

Some nonprofits may feel that it’s important for its policies, practices, and uses of AI to weigh the existential risks associated with AI. After all, more than 350 AI scientists, CEOs, and other experts (including Sam Altman, Dario Amodei, and Bill Gates) signed onto a one-sentence statement organized by the Center for AI Safety:

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

That, in addition to myriad warnings from other AI experts (including here), should get everyone’s attention even if it may be easy for most leaders to dismiss as something too out-of-reach and speculative. But there are some broad societal risks that may capture more people’s attention, such as:

  • Diminishment of human agency and critical thinking; dependency on flawed AI developed for commercial purposes (and greed) and soon to be substantially outside of human controls
  • Massive employment disruptions
  • Environmental harms
  • Threats to democracy
  • Misrepresentative and fraudulent communications (e.g., hallucinations, deepfakes)
  • Concentrations of immense wealth and power
  • Loss of privacy and development of a surveillance state
  • Disproportionate new and continuing harms to marginalized communities

Some or all of these risks may point to how a nonprofit wants to use (or not use) AI, feed (or not feed) data for AI access, and select vendors and AI systems. But the decisions can be complicated and involve trade-offs. For example, if distilling the issue to its simplest form (far too simple for practical use) – where the benefits of a particular decision are rated medium-to-high, and the risks, low-to-medium, the leaders may be prompted to move forward. But there are different types of benefits and risks, and they each have differing relative weights, and individuals may rank them differently. Moreover, making a decision is only part of the equation because the decision still needs to be communicated and implemented while monitoring whether the actual benefits and detriments were accurately forecasted in making the decision.

Difficult stuff. But difficulty cannot lead to inaction. Leaders make difficult decisions using the best information they have reasonably available (and properly investing to make sure they have such information when it’s not readily available).

Documentation

Boards should properly document their processes in staying informed and making thoughtful decisions. This does not mean transcribing all discussions at board meetings (including with the use of AI), but minutes can capture the fact that the board reviewed certain documents and staff presentations and recommendations and spent a reasonable amount of time deliberating over important matters. Supplemental materials might be maintained to reflect how a certain decision was reached. Special care will be required, both in the decision-making and the documentation, if legal compliance or conflict-of-interest issues are raised.

Resources

AI for Nonprofits Resource Hub (NTEN)

How to Create a Generative AI Use Policy (TechSoup)

Getting started on a responsible AI use policy for nonprofits (Meena Das, Candid)

Make AI Your Strategic Thought Partner: Here’s How (David Wheeler, Chronicle of Philanthropy)

Responsible AI: Security and privacy tips for nonprofits (SecureAZ, ASU Lodestar Center for Philanthropy and Nonprofit Innovation)

The 2026 AI Index Report (Stanford Institute for Human-Centered AI)