For What It May Be Worth

As the capabilities of artificial intelligence (AI) rapidly change, the laws regarding AI use likewise adapt. For instance, the New York Surrogate’s Court recently grappled with an expert witness’ use of AI to support his valuation testimony.[1] The expert used Microsoft Copilot—a generative open-source AI model—to calculate the present-day value of $250,000 as if it had been invested in the stock market from December 2004 through January 2021.

To test the AI’s reliability and accuracy, the Court ran a prompt in Microsoft Copilot three times. Each time, the AI model returned a different response despite being fed an identical prompt. Further, despite the expert’s claim that AI use is generally accepted in his field, he could not point to any publications or sources that supported his assertion.

The Court concluded that the admissibility of expert evidence relying on AI—particularly with how fast AI is evolving—should be subject to a hearing to determine whether its use is generally accepted in the relevant field.

While the Connecticut and Massachusetts courts have not yet confronted this issue, it is only a matter of time. According to CoStar, real estate firms are already using AI to analyze leases and value commercial properties.[2] In fact, some brokerage firms are developing their own proprietary AI models.[3] As appraisers incorporate AI into their work, questions will arise concerning the budding technology’s use in the appraisal of real property in Connecticut and Massachusetts property tax appeals.

Pullman’s Property Tax & Valuation attorneys designed an experiment to put AI’s real estate valuation capabilities to test. The experiment involved three of our practitioners entering identical prompts at the exact same time in the same open-source AI model. The practitioners repeated that process across other generative AI models. We conducted the experiment over the course of an afternoon inputting multiple prompts across the models.

The results, similar to those found by the New York Surrogate’s Court, were wildly divergent. In every instance, the AI returned different responses to each practitioner, often based on different source data, despite use of identical prompts. Further, in response to one prompt, a particular AI model relied on phantom sales to support its response; in another instance, the model amended its response after the user suggested, in a follow-up response, that the model’s first answer was less than satisfactory. These results indicate inherent accuracy, reliability, and repeatability issues.

With the rapid development of generative AI, it is tempting to incorporate the emerging technology into a property tax appeal. However, as the New York Surrogate’s Court and our experiment found, AI’s accuracy and reliability in areas as complex as property valuation require further evolution before deployment and tax professionals may want to consider proceeding with caution.

[1] Matter of Weber as Tr. of Michael S. Weber Tr., 220 N.Y.S.3d 620 (N.Y. Sur. 2024).

[2] Andria Cheng, Real Estate Industry Increasingly Relies on Artificial Intelligence, CoStar (2024), https://www.costar.com/article/1520294898/real-estate-industry-increasingly-relies-on-artificial-intelligence.

[3] David Matthews, JLL on How AI Will Affect Commercial Real Estate, LoopNet (2023), https://www.loopnet.com/learn/jll-on-how-ai-will-affect-commercial-real-estate/75780297/.