Article

Solving the small frictions of legal aid referrals with light AI help

Image of Dave Guarino
Image of Dave Guarino
Table of contents

Ever since ChatGPT was released, a lot of discussion has equated AI with “chat interfaces.” That's a limiting perspective. In this post, I’ll share the details of a small internal tool we built recently that makes use of AI in a way completely unrelated to chatbots.

We used a few different AI capabilities (code generation, text processing) to quickly automate a time-intensive task that, a few years ago, we might have shied away from.

Specifically, we built a quick tool to better identify what local legal aid organization can help someone with SNAP problems in a given geography.

When a colleague suggested I write this post to document this work publicly, I hesitated at first, because it seemed too small to be worth writing up.

But reconsidering it, I realized: one of the more immediately valuable uses of AI right now is for automating relatively small (but time-intensive) tasks. So having more publicly documented examples is useful.

At Propel, we provide help with a number of things related to managing safety net benefits. For example, check out this help article on what someone can do if they’ve missed an EBT deposit.

But SNAP and other benefits programs are complicated. Some problems go beyond what we can directly help with.

Legal aid is one of the more important supports of last resort in the SNAP program. One particular way legal aid can help is with fair hearings, also known as appeals. This is the formal process by which someone can assert their disagreement with an agency decision on their case.

However, fair hearings are not used as frequently as you might think. About 21 million households received SNAP in 2021. But for that many cases, there were only about 47,000 fair hearings in that year. That’s about 1 fair hearing for every 450 SNAP cases.

Legal aid assistance can help people with fair hearings in a few ways:

1. Expert Triage: A legal aid org may be able to help avoid a fair hearing, giving information about an alternative way to get a problem fixed. Or, if the problem is not one that a fair hearing will address — like the household realistically being over the income limits — they can also diagnose that. So being helped by legal aid can lead to more “high signal” fair hearings — ones where there is merit to the disagreement.

2. Substantive help with the process: SNAP’s rules are complicated, and fair hearings themselves can be intimidating. I have experienced firsthand people unwilling to appeal a decision because the concept of a “hearing” simply induces too much fear. And research has documented the barriers people face.

For users with more severe or complex SNAP problems, we would ideally want to provide users an easy connection with legal aid in their area. But that can be more complicated than it sounds.

There is no central list for legal aid orgs that help with SNAP problems specifically.

Part of the reason is the reality that this can change across time, as a result of funding and staffing changes.

In this situation, the default approach for many would be to provide generic advice — for example:

Contact your local legal aid for help

But then we are just pushing the friction of finding help onto the person. If we can remove that friction for them, we should.

And we can — there is a rough, implicit step-by-step algorithm for finding legal aid that offers help with SNAP cases:

  • 1. Do a web search for “legal aid” + a state or city name
  • 2. Look around the website to see if they provide help with “SNAP” or “public benefits” or “food stamps”
  • 3. Find the instructions for how someone can get help (sometimes called “intake”)

We definitely don’t want to tell a person to go do all of that. But AI can help here!

We used a few different AI capabilities, strung together, to build a small tool in under an hour. This tool is not perfect, but it provides much better information than what we would provide without it.

I gave Claude (Sonnet 3.5 latest) a high level description of what we would like to do. Note that I provided context, but was not overly directive In fact I explicitly say it should ask me questions if useful:

After a few minutes of back and forth, what we got was a Python tool that did the following:

  1. Used the Google Search API to search for legal aid organizations in each state
  2. Identified the websites of the orgs
  3. Read those websites, identifying if they explicitly offered help with SNAP/public benefits
  4. If they help with SNAP, extract important information off of their websites, for example:
    1. The intake process for a person to actually get help
    2. Phone number
    3. State/counties served
  5. Save the information to a spreadsheet

Steps 3 and 4 (figuring out if the org helps with public benefits, and extracting information) are both steps that explicitly use an AI language model — to put text in and get text out — but the other steps are just regular, procedural code.

So we ended up with something like this:

In less than an hour we have been able to provide likely much more targeted legal aid referrals than we would have been able to before. (And we avoid pushing the work of finding help onto the person.)

What about risks?#what-about-risks

It’s certainly possible for this tool to get things wrong. It might miss an organization, or it might miscategorize an organization as providing public benefits help. Those errors are similar to what might happen if we asked a person to do this research manually, though probably with some marginal error increase using AI.

However, providing a helpful response with some margin of error can better than strictly accurate (but unhelpful) guidance.

Here the impact of an error on the user is low. In the worst case, they likely make contact with an org in their area that can provide a handoff if they do not help with SNAP.

It’s useful to compare that to what will most likely happen if we just say “contact your local legal aid”:

  • Many people simply stop there, because the next step is too vague
  • Those who do take action are most likely to do something like search “legal aid” on Google
    • So they replicate our automated process, but without knowing to check if that org helps specifically with public benefits
  • We’ve shifted the burden of this research onto the person

The potential negative impact here is quite different from other problems we might see as having a similar shape.

For example: LIHEAP application dates can vary across the country. But if we tried the same “fuzzy AI automation” approach to identify that, the downside impact from a failure could be much greater: someone might miss out on the benefits altogether.

That is a far more adverse impact on users, and it illustrates why calibrating to the impact an error has is important in deciding when and where to use these kinds of tools.

How is this type of AI usage different?#how-is-this-type-of-ai-usage-different

It differs in a few meaningful ways:

1. We are not using AI as an “oracle” – a magic box that can answer any question

A lot of people project a mental model — again, likely due to the chat UX being pervasive — that modern AI tools are powerful question-answering machines.

Here we are doing something rather different: we have a rough sense of how to go about answering a question, and we’re using AI to automate that with capabilities with much lower failure rates (code generation, narrow text processing of web page content).

This sort of use is, by its nature, much safer and less prone to error than the “oracle” usage.

2. Nothing here has anything to do with chat

Chatbots are one interface for this capability set. Here, no user has a chat interaction. Rather we are building up static content with automating a background research process.

Many problems in the social safety net are similarly shaped to this#many-problems-in-the-social-safety-net-are-similarly-shaped-to-this

It’s also worth noting that this shape of a problem — where there are lots of local variations on a service, but it’s too time-intensive to manually map them and keep it up-to-date — is a surprisingly common one in safety net benefits, social services, and related domains.

Problems like:

  • Finding a local food bank or pantry that is open
  • Finding city-level housing assistance programs
  • Identifying local community based resources specific in a place
  • Tracking state level legislation

…all have a similar shape.

And there have been many efforts to try to tackle this problem with different approaches (e.g. data standardization, open source web scrapers)

But new tooling that can do in a fuzzy way what until recently required lots of hard coding or manual effort — and can even empower people with very limited programming backgrounds to make use of this tooling — changes how many useful services can be built for problems of this shape.