Image Summary

A multilingual AI mobile app that extracts text from document images, summarizes it in the user’s language, and converts it into speech, all in one seamless flow.

Highlights

• Multilingual OCR + summarization (Pashto, Dari, English)
• AI-powered text understanding and speech output
• Fully client-side React Native app
• Delivered MVP in 3 weeks

Quick Facts

• Industry: Accessibility / Language Assistance
• Platform: iOS & Android (Mobile)
• Services: UI/UX, Mobile Development, AI Integration
• Tech Stack: React Native, Redux, OpenAI APIs

Background

Understanding English documents remains a significant challenge for many Pashto and Dari speakers. In everyday situations—official documents, educational material, or written instructions—users often rely on human assistance to translate and explain content.

The client set out to create a simple, mobile-first solution that would allow users to:

• Capture a document as an image
• Instantly understand its meaning in their own language
• Listen to a concise spoken summary without reading long text

The goal was not to build a full document management system, but a focused accessibility tool: fast, lightweight, and easy to use.

Challenge

Business & User Challenges

• Heavy dependence on human translators or helpers
• Existing OCR tools focused on extraction, not understanding
• Poor support for Pashto and Dari in mainstream apps
• High cognitive load when reading long translated text

Technical Challenges

• Achieving acceptable OCR accuracy for document images
• Managing AI API costs under strict budget constraints
• Ensuring usable text-to-speech quality for non-Latin languages
• Delivering a smooth experience with no auth, no storage, no backend

Solution

Stackup Solutions designed and delivered a stateless, AI-powered mobile application focused on one core workflow: Image → Understanding → Audio.

Architecture & Technical Decisions

• Client-only architecture to reduce complexity and cost
• Server-based OCR service for reliable text extraction
• OpenAI Chat Completions (gpt-4o-mini) for summarization
• OpenAI Text-to-Speech (gpt-4o-mini-tts) for spoken summaries
• Redux for predictable, minimal state management
• Image Crop Picker to improve OCR accuracy

AI Logic

• Extracted OCR text is sent directly to the LLM
• Model generates a casual, easy-to-understand summary
• Summary is produced in the user-selected language
• Summary text is converted to speech

To control hallucinations and cost:

• Only raw OCR text is sent to the model
• Summaries are intentionally concise
• No post-processing or enrichment is applied

Feature Breakdown

• Image upload and cropping
• Language selection (OCR, summary, speech)
• AI-generated text summaries
• Audio playback (play / pause)
• Clean, minimal UI for first-time users

Core User Flow

1. User opens the app
2. Selects preferred language
3. Uploads and crops a document image
4. OCR extracts text from the image
5. AI generates a short summary in the selected language
6. Summary is converted to speech
7. User listens to the spoken explanation

The entire experience is completed in a single session, with no sign-up and no stored data.

Implementation Process

Phase 1: Rapid MVP Definition

• Clarified accessibility-first scope
• Focused on documents and summaries only
• Removed non-essential features (auth, history, dashboards)

Phase 2: AI & Mobile Integration

• Integrated OCR service with image preprocessing
• Designed cost-efficient summarization prompts
• Implemented TTS playback with minimal controls

Phase 3: Optimization & QA

• Tuned OCR flow using image cropping
• Improved Pashto/Dari text rendering
• Added toast-based error handling

Timeline: 3 weeks
Team: React Native developers
Scope: UI/UX, Mobile Development, AI Integration

Results & Impact

Although still in MVP stage, the solution delivered clear value:

• Improved OCR success rates through image cropping
• Reduced reliance on human assistance
• Faster comprehension via spoken summaries
• Validated Pashto/Dari-focused AI accessibility use cases

The app established a foundation for future expansion into:

• Multi-page documents
• Enhanced voice controls
• Offline or low-bandwidth optimization
• Broader language support