Articles

Is Your Organization AI Ready? (Part II)

Is Your Organization AI Ready?

By Cheryl Ahrens Young, IGP, CSM, CSPO, CTT+, ERMm, ECMp 
[email protected]

An AI is only as good as the information in its LLM.  

AI developers assume that the large language models built from existing databases and systems’ of record metadata are accurate, factual, reliable and have data integrity.  My experience in business process automation with learning servers has taught me that assumptions lead to “ass: u & me”.   

You will often see a disclaimer in content created by commercial “free” AI that the information provided may not be accurate. 

What is the root cause for the inaccuracies in an AI generated document?  For AI hallucinations? 

ROT – Redundant, Obsolete and Trivial records fed to the LLM.  Added to that, lack of integrity in the information – abbreviations, mis-spellings, homonyms that mean entirely different things (accept vs except), and acronyms that are industry specific (CRM could be Certified Records Manager or Customer Relations Management).

How do you clean up the ROT and improve the integrity of the LLM?

A legally defensible, robust records and information management program which addresses not only retention but integrity, accuracy and accountability.  This all starts with a current records retention schedule that is then applied to the systems of record, shared drives and any other source of information that builds the LLM.  In addition, processes and procedures outline how to name records and enter consistent data into the systems of record.

Who knows about records retention schedules and naming conventions?

Records and Information Managers! 

The question is – how do we get a seat at the AI table?  

Where I’ve had success in getting the technical team to listen is inviting them to a lunch and learn meeting and talking about RIM and knowledge management.  If your company follows Agile or Scrum project methodologies, ask to be part of the team to develop the customer stories.  Your story might revolve around an AI identifying duplicate and near duplicate copies across shared drives and systems of record to reduce the ROT to then provide the best possible information for a iterative process improvement of the information fed to the LLM.

Is Your Organization AI Ready?

Is Your Organization AI Ready?

By Cheryl Ahrens Young, IGP, CSM, CSPO, CTT+, ERMm, ECMp 
[email protected]

The more things change, the more they remain the same.  The graph below is from 2008, when learning servers were gaining hold in streamlining workflow processes, like accounts payable processing of invoices. 

At INFOCON this year, this point was driven home, again, by the keynote speakers.  An AI is only as good as the information in its LLM.  In every project I worked on in the following seventeen years, not one database was 100% accurate.  Errors ranged from using acronyms and abbreviations to fields missing entirely.  For one project, I was given an organization’s glossary of acronyms, the meanings of which were dependent on the division that entered the data.

 Why did this happen? Managing the database software was under the purview of IT and they left it up to the business units to ensure data accuracy.  How accurate are you when you are on deadline or it’s Friday afternoon before a 3-day weekend? Most non-manufacturing organizations do not follow either 6-Sigma or Lean guidelines to reduce defects and there are not any data checks in place beyond a 10% random sampling after a migration.

 So, the first set of tasks in one of my process improvement projects, which AI can definitely be, was to standardize and correct the data.

 Agentic AI assumes that the large language models built from existing databases and systems of record metadata are accurate, factual, reliable and have data integrity.  For those of us in the field of records and information management, information governance and project management, we know that an assumption leads to “ass: u & me”.   This is before you add in the comprehension of intent in unstructured documents and notes which have also been used to “teach” AIs, like ChatGPT and Grok.  

 For this reason, you will often see a disclaimer in content created by AI that the information provided may not be accurate.  There was preliminary study that it is taking, on average, two hours for a person to correct a document that AI has created because they have to verify the facts, like citations, before it can be released.

 What can we do?  Use Microsoft’s own recommendations for a file plan and retention labels to get a seat at the table of your AI team!  (Troubleshoot Copilot Tuning document generation | Microsoft Learn)

 If you didn’t attend InfoCon this year, the keynote speakers presentations can be found here: Troubleshoot Copilot Tuning document generation | Microsoft Learn

 And keep checking the OCARMA website for information on the upcoming 4^th Annual California Chapters Summer conference!