Home Original page

GitHub - jdot274/ControlX: macOS menubar control app with AI agent, dev workflows, and system monitoring

A powerful macOS menu bar app that allows you to select multiple items at once and perform batch operations, plus zoom into any area of your screen.

Features

🎯 Context-Aware Actions (v1.3!)

  • Smart dropdowns for every action: Upload, Share, Open With, Save To
  • 6 cloud/share options: Dropbox, iCloud, Google Drive, Email, Messages, AirDrop
  • Multiple editor choices: TextEdit, VS Code, Cursor, Preview, Photoshop, + custom
  • Quick locations: Documents, Desktop, Downloads, + custom picker
  • Keyboard shortcut: ⌘+A

✏️ Quick Note (v1.3!)

  • Translucent floating notepad - see through to your work!
  • Type quick thoughts and instantly:
    • 💾 Save to any location
    • 📄 Make Document (auto-save to ~/Documents)
    • 📤 Share (dropdown: Email, Messages, AirDrop, Dropbox, iCloud)
    • 📋 Copy to clipboard
  • Auto-naming based on first few words
  • Keyboard shortcut: ⌘+N

🔵 Window Highlighter (v1.3!)

  • Hover over any window to see blue glowing edges
  • Contextual action buttons appear:
    • 📸 Screenshot just that window
    • 📹 Record the window
    • 🔄 Minimize it
    • 📌 Pin always on top
    • ↔️ Quick resize
  • Shows app name and window title
  • Keyboard shortcut: ⌘+W

📁 File Capture Mode

  • Revolutionary: Turn screenshots into interactive file browsers!
  • Select any area showing files/folders
  • AI-powered OCR detects all files
  • Creates professional Finder-style window
  • Click files to open them directly
  • Search, filter, and export file lists
  • Keyboard shortcut: ⌘+C

🔍 Find Window or App

  • Instant search for any open window or installed application
  • Type to search - results filter in real-time
  • Perfect for finding specific windows when you have many open (e.g., multiple Chrome windows)
  • Shows app icons and window titles
  • One-click to switch to window or launch app
  • Keyboard shortcut: ⌘+F

🎯 Box Selection Mode

  • Click and drag to draw a selection box over multiple items (checkboxes, buttons, etc.)
  • Automatically detect and click/check all items within the selected area
  • Perfect for bulk operations on checklists and forms

🔎 Zoom Mode

  • Select any area of your screen to zoom in
  • Use scroll wheel or +/- keys to adjust zoom level (0.5x to 10x)
  • Perfect for reading small text or examining details

💚 Status Indicator

  • When enabled, a green "ControlX is ON" indicator appears at the top center of your screen
  • Pulsing animation shows the app is active
  • Menu bar icon changes to show active status

Installation

Requirements

  • macOS 11.0 or later
  • Xcode Command Line Tools (for building)

Building the App

  1. Open Terminal and navigate to the ControlX folder:
cd /Users/joeywalter/Desktop/ControlX
  1. Make the build script executable:
  1. Run the build script:
  1. The app will be created in the build folder as ControlX.app

First Time Setup

  1. Double-click ControlX.app to launch it
  2. Look for the checkmark icon in your menu bar (top right)
  3. Grant permissions when prompted:
    • Accessibility: Required to detect and click UI elements
    • Screen Recording: Required for zoom feature
  4. Go to System Settings > Privacy & Security:
    • Under Accessibility, ensure ControlX is enabled
    • Under Screen Recording, ensure ControlX is enabled
  5. Restart the app after granting permissions

Usage

Enabling ControlX

  1. Click the ControlX icon in the menu bar
  2. Select "Enable ControlX"
  3. A green indicator will appear at the top center of your screen

Quick Actions with Dropdowns (v1.3!)

  1. Click the menu bar icon and select "⚡ Quick Actions" (or press ⌘+A)
  2. Hover over any file or select text
  3. See context-aware actions with dropdowns:
    • 📤 Upload/Share to ▾ → Choose: Dropbox, iCloud, Google Drive, Email, Messages, AirDrop
    • 📂 Open with ▾ → Choose: TextEdit, VS Code, Cursor, Preview, Photoshop, or custom
    • 💾 Save to ▾ → Choose: Documents, Desktop, Downloads, or pick location
  4. Click dropdown to select your preferred app/location

Quick Note (v1.3!)

  1. Click the menu bar icon and select "✏️ Quick Note" (or press ⌘+N)
  2. Type your thought in the translucent window
  3. Choose an action:
    • 💾 Save → Pick custom location
    • 📄 Document → Auto-save to ~/Documents with smart naming
    • 📤 Share → Dropdown with 5 methods (Email, Messages, AirDrop, Dropbox, iCloud)
    • 📋 Copy → Copy to clipboard

Pro Tip: The window is translucent so you can see your work underneath!

Window Highlighter (v1.3!)

  1. Click the menu bar icon and select "🔵 Window Highlighter" (or press ⌘+W)
  2. Click "Enable Highlighting"
  3. Move mouse over any window → blue glow appears
  4. Action buttons show up:
    • Screenshot, Record, Minimize, Pin, Resize
  5. Click an action to execute

File Capture Mode

  1. Click the menu bar icon and select "📁 File Capture Mode" (or press ⌘+C)
  2. Click the "Enable Capture" button (turns green)
  3. Select any area on your screen showing files or folders
  4. Wait 2-5 seconds for AI analysis
  5. View all detected files in a professional window
  6. Search, filter, or click any file to open it!

Example: Screenshot of Downloads folder with 50 files:

  • Press ⌘+C
  • Enable capture (button turns green)
  • Select the screenshot area
  • Get interactive window with all 50 files
  • Search for "invoice" → Click to open!

Find Window or App

  1. Click the menu bar icon and select "Find Window or App" (or press ⌘+F)
  2. Type to search for any window or application
    • Searches window titles and app names
    • Results show app icons and status (OPEN or LAUNCH)
  3. Click a result or press Enter to switch to that window
  4. Press ESC to close the search window

Example: If you have many Chrome windows and one is ESPN:

  • Press ⌘+F
  • Type "espn"
  • Click the ESPN result → switches to that window!

Box Selection Mode

  1. Make sure ControlX is enabled
  2. Click the menu bar icon and select "Box Selection Mode" (or press ⌘+S)
  3. Click and drag to draw a selection box over items you want to select
  4. Release to complete the selection
  5. Choose an action:
    • Click All Items: Click all buttons and checkboxes in the area
    • Check All Checkboxes: Only check unchecked checkboxes
    • Cancel: Do nothing

Zoom Mode

  1. Make sure ControlX is enabled
  2. Click the menu bar icon and select "Zoom Selection Mode" (or press ⌘+Z)
  3. Click and drag to select the area you want to zoom into
  4. Use the scroll wheel or +/- keys to adjust zoom level
  5. Press ESC to exit zoom mode

Keyboard Shortcuts

  • ⌘+A: Quick Actions (context-aware dropdowns)
  • ⌘+N: Quick Note (translucent notepad)
  • ⌘+W: Window Highlighter (hover to see blue glow)
  • ⌘+C: Open File Capture Mode
  • ⌘+F: Open Find Window or App
  • ⇧⌘+E: Smart Explainer Mode
  • ⌘+E: Toggle ControlX on/off
  • ⌘+S: Start Box Selection Mode
  • ⌘+Z: Start Zoom Mode
  • ESC: Cancel current operation or exit mode
  • + or =: Zoom in (when in zoom mode)
  • -: Zoom out (when in zoom mode)
  • Enter: Switch to selected window (in search) or open file

How It Works

Box Selection

ControlX uses macOS Accessibility APIs to:

  1. Detect UI elements (buttons, checkboxes) within your selected area
  2. Sample points in a grid pattern across the selection
  3. Identify clickable elements at each point
  4. Perform click actions on all detected elements

Zoom Feature

ControlX captures a screenshot of your selected area and displays it in a full-screen overlay where you can zoom in and out for detailed viewing.

Troubleshooting

App won't detect or click items

  • Make sure you've granted Accessibility permissions
  • Restart the app after granting permissions
  • Some apps may have protected UI elements that can't be automated

Zoom feature not working

  • Ensure Screen Recording permission is granted
  • Check System Settings > Privacy & Security > Screen Recording

App doesn't appear in menu bar

  • The app runs as a menu bar-only application (LSUIElement)
  • Look for the checkmark icon in the top-right menu bar
  • Try restarting the app

Permission dialogs don't appear

  • Manually go to System Settings > Privacy & Security
  • Add ControlX to Accessibility and Screen Recording manually

Technical Details

  • Built with Swift and AppKit
  • Uses macOS Accessibility APIs for UI element detection
  • Menu bar app (no dock icon)
  • Supports multiple displays
  • Works on Apple Silicon (arm64) Macs

Privacy

ControlX runs locally on your Mac and does not:

  • Send any data to external servers
  • Track your usage
  • Access the internet
  • Store any personal information

All operations are performed locally using macOS system APIs.

License

This is a custom-built app for personal use.

Support

For issues or questions, check that:

  1. All required permissions are granted
  2. You're running macOS 11.0 or later
  3. The app has been properly signed with codesign

Enjoy using ControlX! 🚀