How Shmor's AI reads a screenshot or a voice note, and what it won't guess

Shmor's AI turns what you send (a screenshot, a photo of a card or badge, a voice note, a LinkedIn link or a line of text) into a CRM record with a fixed set of fields. It is built to fill a field only when the answer is in front of it, to leave everything else empty, and to ask you for the gaps that matter. This post explains, step by step, what happens to a capture, what is extracted, the rules that stop it guessing, and where it still gets things wrong.

The short version

• What reads it: OpenAI's models, through OpenAI's API. Voice notes are transcribed first, then everything is read together.
• What it extracts: the person, their company, how and where you met, a short note, a follow-up, and whether it sounds like a deal.
• What it will not do: invent an email, guess a company website, look people up on the web, or contact anyone.
• What it asks you: how you met, and whether and when to follow up, when you did not say.
• What it keeps: Shmor never stores your images or audio.

Step 1: voice notes become text

If you send a voice note on Telegram, or attach a recording to an email you send to in@emails.useshmor.com, Shmor first sends the audio to OpenAI's speech-to-text service and gets a transcript back.¹ You see "Transcribing" in Telegram while it happens.

Two things help the transcript:

• Your word list. In Settings, Words to listen for in voice notes holds the names transcription tends to mishear: your company, products, events, unusual surnames. Shmor passes them to the transcription as hints.
• Your reply card. The card Shmor sends back includes the start of what it heard, so a misheard name is visible straight away.

If there is no speech it can make out, Shmor says so and stops. Nothing is saved, and the message does not use a capture.

Step 2: one read of everything you sent

Shmor then makes one request to an OpenAI model with everything it knows about this capture:

• the images you sent, at high detail;
• your text, plus the voice transcript;
• today's date, in your time zone, so "next Thursday" becomes a real date;
• the data Apollo returned, if you sent a LinkedIn link and added your own Apollo key;
• a short description of your business and your list of services, if you set them up, so it can judge whether a conversation sounds like an opportunity;
• on an update, the existing contact and the bot message you replied to.

The model must answer in a fixed structure. It cannot return free-form text or extra fields. That structure is the list below.

What it extracts

On Pro, you can also map up to 10 of your own Attio or HubSpot person fields, each with a sentence telling Shmor what to put there. Those fields follow a stricter rule still: filled only when the screenshot, your words or the Apollo data clearly state the value, never inferred from a job title or a company name.

The rules that stop it guessing

The model works under a short set of instructions. In plain English:

1. Only facts that are there. Output only what is visible in the images, written or said by you, or in the Apollo data. Never invent names, emails, phone numbers, websites or companies. Leave a field empty when unknown.
2. You win over the database. If your screenshot or your words disagree with Apollo, your version is used.
3. The person is never you. On a screenshot of your own LinkedIn feed, the person is the profile being captured.
4. Ignore the clutter. Interface elements, adverts and "People also viewed" panels are not the person.
5. Dates are real dates. Relative dates are converted using today's date.

Instructions reduce mistakes but cannot rule them out, so the output then goes through plain code checks before anything reaches your CRM:

• email addresses are cleaned, lower-cased, de-duplicated, and dropped if they are not well-formed, including the placeholder text some tools show instead of a hidden email;
• phone numbers with fewer than 7 digits are dropped;
• a LinkedIn URL is kept only if it is a real profile link, and a link you actually sent always beats one the model read;
• a company website is kept only if it is a valid domain, and never a personal email provider such as Gmail, Outlook, iCloud or BT Internet. If no website was shown, the domain of a work email address is used instead.

Finally, there must be a person. If the model found no name, no email and no LinkedIn link, Shmor replies that it could not find a person and saves nothing, and that message does not use a capture.

How it asks for what is missing

Shmor saves what it can straight away, then asks about the two things a useful record needs and a screenshot rarely shows:

• How did you meet? If you did not say, the card offers buttons: Networking event, Conference, Intro, Inbound, Online or Other.
• Follow up? If you did not say, and the person has no open follow-up already, the card offers your default number of days, 1 week, 2 weeks or No follow-up. After a timeframe, it asks what the follow-up is about.

Anything else, you add by replying to the card in your own words: "her email is sam@northwind.co.uk", "met at SaaStock", "make it the 14th". On a reply, the model is told to return only what you are adding or correcting, and to leave the rest of the record alone. A name changes only when you give both first and last name, so a reply about something else can never rename the person by accident.

Sending pending lists up to 10 people who still have an open question.

Deals: suggested, never created

If you have described your business and listed your services, the model also judges whether the capture sounds like a potential sale, partnership or paid engagement, and which of your services it relates to. On Pro and the free trial, the card then shows Possible deal with Create deal and Not a deal buttons. Nothing is created until you tap. See tracking deals from the people you meet.

Where it still gets things wrong

Being honest about the limits helps you work around them:

• Handwriting and stylised fonts. Handwritten numbers on cards and decorative logos are read less reliably.
• Unusual names in voice notes. Names the transcription has never heard can come out phonetically. Spell them, add them to your word list, or send a screenshot.
• Noisy rooms. Background music and crosstalk reduce transcript quality.
• Out-of-date profiles. Shmor reads the screenshot, not the world. If a profile still shows last year's job, that is what is saved.
• Ambiguous context. "Follow up after the holidays" depends on which holidays you mean. A specific day is safer.
• Several people in one message. Each capture saves one person. Send one per message.
• PDFs. Attached PDFs are not read yet; send a screenshot of the page instead.

Shmor also does not search the web, does not scrape LinkedIn, and never messages the people you capture.

What is sent, and what is kept

• Sent to OpenAI: the images, audio and text of each capture, and what is extracted from them. OpenAI processes this in the United States, under the UK Addendum to the EU Standard Contractual Clauses, as listed on our sub-processors page.
• Training: under OpenAI's API data controls, API data is not used to train its models by default, and abuse monitoring logs are kept for up to 30 days.² Shmor never uses your messages to train models either.
• Kept by Shmor: never your images or audio. With ShmorCRM, the records and the text of each capture, including voice transcripts, until you delete them. With Attio or HubSpot connected, Shmor's own database keeps only record IDs and message IDs, and the contact lives in your CRM.

Our security page and privacy notice have the full detail.

Getting the best out of it

• Send one person per message.
• Add one sentence of context: how you met, what they need, when to follow up.
• For voice, spell unusual names and keep it under 2 minutes.
• For screenshots, include the name and company in the frame.
• Glance at the card before the next conversation, and correct by reply.

Step-by-step guides: sending voice notes to your CRM and turning screenshots into CRM contacts.