V1.0.13: Staging

.env.example

@@ -0,0 +1,6 @@
+AIRBYTE_SUPABASE_PROXY_URL = "https://xxxxxx/functions/v1/xxxxx/airbyte";
+AIRBYTE_LOCAL_PROXY_URL = "/api/airbyte";
+AIRBYTE_API_BASE_URL = "https://api.airbyte.com/v1";
+GROQ_SUPABASE_PROXY_URL = "https://xxxxxxx/functions/v1/xxxxx/groq";
+GROQ_LOCAL_PROXY_URL = "/api/groq";
+GROQ_API_BASE_URL = "https://api.groq.com/v1";

README.md

@@ -4,22 +4,34 @@
 
 <b>Transform your business with <code>Analysr</code></b>
 
-## ⚡ One Liner
+## ⚡ Speedy Summary
 This is my submission for <code>Airbyte-Motherduck Hackathon - December 2024 - January 2025</code>
 
-For all you speedy folks out there, here’s the summary:
+Here’s a speedy summary:
 
 - **1.0.0**  
   - With your customer reviews in Motherduck, along with your chosen business stack and areas of interest, Analysr is ready to dish out some insightful analytics. To sweeten the deal, Groq is also integrated to help you navigate all your growth phases.
   - Your analytics lineup features Aspect Analysis, a Word Sentiment Heatmap (for those feelings), Advanced Text Analysis, Groq Business Analytics, Keyphrase Analysis, and a handy Competitor Comparison.
+  - Check it out at:
+    - <code>[https://growwithanalysr.web.app/](https://growwithanalysr.web.app/) - Production</code>
+    - <code>[https://growwithanalysr.vercel.app/](https://growwithanalysr-staging.vercel.app/) - Experimental, for new features</code>
 
 ## 🏗️ Architecture
 ![image](https://github.com/user-attachments/assets/0abe96f6-414a-42d2-aa0d-d0950a7da194)
 
 
+## ❓ Why Analysr
+
+- **Scale Beyond Regular AI Capabilities:** Traditional AI systems, like ChatGPT, struggle to handle extensive datasets (e.g., 65,000+ records) effectively. Analysr bridges this gap.
+- **Seamless Motherduck, Airbyte, Groq Integration:** Thanks to Motherduck wasm client, Airbyte's API and Groq SDK.
+- **Data-Driven Insights:** By combining AI with visualization tools, Analysr allows users to uncover trends, anomalies, and actionable insights quickly and intuitively.
+- **User-Friendly Visualization:** Visual AI integration transforms raw data into understandable and compelling graphics, enabling better decision-making.
+- **Streamlined Process**: Reduces reliance on multiple tools by offering an all-in-one platform for schema analysis and visualization.
+
+
 ## 🚶Walkthrough
 
-1) To obtain customer review insights, sync your data to Motherduck with the schema: <code>{ "review_text": "string", "stars": "number" }</code> (More schemas will be supported soon). We recommend using Airbyte due to its extensive list of sources and seamless data movement. ![image](https://github.com/user-attachments/assets/415aece5-6594-4649-8d84-ec2fa1707988)
+1) To obtain customer review insights, sync your data to Motherduck with the schema: <code>{ "review_text": "string", "stars": "number" }</code> (More schemas support are in the future roadmap). We recommend using Airbyte due to its extensive list of sources and seamless data movement. ![image](https://github.com/user-attachments/assets/415aece5-6594-4649-8d84-ec2fa1707988)
 ![image](https://github.com/user-attachments/assets/00bf63f5-952f-491a-9ffd-0241d2e2bfd2)
 2) Visit the Analysr website at (growwithanalysr.web.app) and click on the "Get Started Now" button for onboarding.
 ![image](https://github.com/user-attachments/assets/95da4b69-29bb-4c88-9433-19865bc72093)
@@ -37,14 +49,18 @@ For all you speedy folks out there, here’s the summary:
 8) Finally, input your area of interest for insights, such as customer satisfaction, and click "Continue to Dashboard."![image](https://github.com/user-attachments/assets/3c938fa2-a862-4ba6-b06e-b67bb139e71f)
 9) Wait a few seconds until all queries are executed and visualized.
 ![image](https://github.com/user-attachments/assets/cf22aa51-cdb2-4e3f-99d6-ef93bf8f8c45)
-10) Voilà! Your dashboard will be ready, featuring all Analysr's capabilities to support your next big step!
+10) Voilà! Your dashboard will be ready, featuring all of Analysr's capabilities to support your next big step!
 ![image](https://github.com/user-attachments/assets/1ae1427d-c315-4e02-ac75-158e3cb14d61)
 
-Need dataset and example method to test?
-1. Hugging face dataset URL which I used, https://huggingface.co/datasets/Yelp/yelp_review_full
-2. Import it to motherduck via airbyte (Set huggingface as source and motherduck as destination)
-3. Get Groq token at, https://console.groq.com/keys
-4. Click on continue to dashboard! That's it. Please try yourself, its fun!
+**Need a dataset and one example method to test?**
+1. Hugging face dataset URL which I used - https://huggingface.co/datasets/Yelp/yelp_review_full
+2. Import it to Motherduck via Airbyte (Set huggingface as source and Motherduck as destination) OR attach using my share link
+```bash
+-- Run this snippet to attach the database
+ATTACH 'md:_share/my_db/de60469b-3a05-4d74-bf63-4c1549dd55b6';
+```
+3. Get a Groq token at, https://console.groq.com/keys
+4. Click on Continue to the dashboard! That's it. Please try it yourself, it's fun!
 
 ## ✨ Features
 
@@ -57,7 +73,7 @@ Need dataset and example method to test?
 
 ## 🛠️ Technology Stack
 
-- **Frontend**: React, TypeScript, Tailwind CSS
+- **Frontend**: React, TypeScript, Tailwind CSS, Vite
 - **Analytics**: MotherDuck (DuckDB), GROQ AI
 - **Data Integration**: Airbyte
 - **Visualization**: Recharts
@@ -67,13 +83,15 @@ Need dataset and example method to test?
 - **Proxy**: Supabase edge functions
 - **CI/CD**: GitHub Actions for automated deployment
 
-## Future roadmap
+**Declarations:** For development, the VSCode code editor, Codeium AI helper extension, and suggestions from ChatGPT were used.
+
+## 🔮 Future roadmap
 
-- **Microservice for generating queries**: Currently all queries for analytics are highly coupled with code, seperation of concerns to microservice
-  - [x] Create mock express server and deployed as supabase functions
-  - [ ] Separate DuckDB queries for as an api call
+- **Microservice for generating queries**: Currently all queries for analytics are highly coupled with code, separation of concerns to microservice
+  - [x] Create express server proxy and deploy as superbase functions
+  - [ ] Separate DuckDB queries as an API call response
   - [ ] Enhance microservice with GPT Wrapper
-  - [ ] Enhance business insights from Groq: Currently it hallucinates as the mixtral model is not powerful (Requires funding) 
+  - [ ] Improve business insights from Groq: At present, it produces some inaccuracies due to the limitations of the open-source <code>mixtral</code> model, which lacks the necessary funding to enhance its capabilities.
 
 ## 🚀 Getting Started
 

package.json

@@ -1,7 +1,7 @@
 {
   "name": "analysr",
   "private": true,
-  "version": "1.0.10",
+  "version": "1.0.13",
   "type": "module",
   "scripts": {
     "dev": "vite",

src/components/dashboard/Analytics/StatGrid.tsx

@@ -31,7 +31,7 @@ export default function StatGrid({ analyticsData }: StatGridProps) {
       color: "purple",
     },
     {
-      title: "Competitor Comparison",
+      title: "Competitive Advantage",
       value: `${
         analyticsData.competitorComparison > 0 ? "+" : ""
       }${analyticsData.competitorComparison.toFixed(1)}%`,

src/components/dashboard/DashboardView/DashboardContent.tsx

@@ -15,13 +15,15 @@ interface DashboardContentProps {
   stack?: string;
   substack?: string;
   groqToken?: string;
+  interests?: string;
 }
 
 export default function DashboardContent({
   analyticsData,
   stack,
   substack,
   groqToken,
+  interests,
 }: DashboardContentProps) {
   if (!analyticsData || !analyticsData.totalReviews || !stack || !substack) {
     return <NoDataFallback />;
@@ -46,6 +48,7 @@ export default function DashboardContent({
           stack={stack}
           substack={substack}
           groqToken={groqToken}
+          interests={interests}
         />
       </Suspense>
 

src/components/dashboard/GPTInsights/BusinessInsights.tsx

@@ -12,13 +12,15 @@ interface BusinessInsightsProps {
   data: ProcessedAnalytics;
   stack: string;
   substack: string;
+  interests?: string;
   groqToken?: string;
 }
 
 export default function BusinessInsights({
   data,
   stack,
   substack,
+  interests,
   groqToken,
 }: BusinessInsightsProps) {
   const [selectedModel, setSelectedModel] = useState("mixtral-8x7b-32768");
@@ -27,7 +29,8 @@ export default function BusinessInsights({
     substack,
     data,
     groqToken,
-    selectedModel
+    selectedModel,
+    interests
   );
 
   const handleModelChange = (modelId: string) => {

src/components/onboarding/DataSelectionStep.tsx

@@ -59,7 +59,6 @@ export default function DataSelectionStep({
 
         setDatabases(dbList);
       } catch (err) {
-        console.error("Error fetching databases:", err);
         setError("Failed to fetch databases");
       } finally {
         setLoading(false);
@@ -87,7 +86,6 @@ export default function DataSelectionStep({
           .sort();
         setTables(tableList);
       } catch (err) {
-        console.error("Error fetching tables:", err);
         setError(err instanceof Error ? err.message : "Failed to fetch tables");
       } finally {
         setLoading(false);
@@ -136,7 +134,6 @@ export default function DataSelectionStep({
           }
         }
       } catch (err) {
-        console.error("Error fetching row count:", err);
         setError(
           err instanceof Error
             ? err.message

src/components/onboarding/OnboardingForm.tsx

@@ -91,7 +91,6 @@ export default function OnboardingForm() {
 
       navigate("/dashboard", { state: finalData });
     } catch (error) {
-      console.error("Form submission error:", error);
       setValidationError(
         error instanceof Error ? error.message : "An error occurred"
       );

src/components/pages/Dashboard.tsx

@@ -77,15 +77,18 @@ export default function Dashboard() {
   return (
     <div className="min-h-screen bg-gradient-to-b from-gray-900 to-gray-800 text-white p-6">
       <div className="space-y-6 mb-8">
+      <div className="max-w-6xl mx-auto">
         {isMockData && <MockDataBanner />}
         <StatusIndicator {...statusState} />
+        </div>
       </div>
 
       <DashboardContent
         analyticsData={analyticsData}
         stack={businessData.stack}
         substack={businessData.substack}
         groqToken={businessData.groqToken}
+        interests={businessData.interests}
       />
     </div>
   );

src/components/welcome/HeroContent.tsx

@@ -24,4 +24,4 @@ export default function HeroContent() {
       </motion.p>
     </div>
   );
-}
+}

src/components/welcome/WelcomeBackground.tsx

@@ -40,4 +40,4 @@ export default function WelcomeBackground() {
       <div className="absolute inset-0 bg-[url('https://images.unsplash.com/photo-1557683316-973673baf926?auto=format&fit=crop&q=80')] opacity-[0.02] bg-cover bg-center" />
     </div>
   );
-}
+}

src/components/welcome/WelcomeHero.tsx

@@ -1,8 +1,8 @@
-import { motion } from 'framer-motion';
-import { BarChart3 } from 'lucide-react';
-import HeroContent from './HeroContent';
-import FeatureGrid from './FeatureGrid';
-import { welcomeScreenData } from './welcomeScreenData';
+import { motion } from "framer-motion";
+import { BarChart3 } from "lucide-react";
+import HeroContent from "./HeroContent";
+import FeatureGrid from "./FeatureGrid";
+import { welcomeScreenData } from "./welcomeScreenData";
 export default function WelcomeHero() {
   return (
     <div className="flex flex-col justify-center p-6 md:p-8 lg:p-16 lg:border-r border-white/10 min-h-screen lg:min-h-0">
@@ -40,18 +40,12 @@ export default function WelcomeHero() {
         initial={{ opacity: 0 }}
         animate={{ opacity: 1 }}
         transition={{ delay: 0.8 }}
-        style={{
-          display: "flex",
-          justifyContent: "center",
-          flexDirection: "row",
-          alignItems: "center",
-        }}
-        className="sm:hidden w-20 mt-8 p-0 md:p-2 bg-gradient-to-r from-blue-500/5 via-purple-500/5 to-blue-500/5 backdrop-blur-sm border border-white/5 rounded-xl"
+        className="hidden lg:flex items-center justify-center w-20 mt-8 p-0 md:p-2 bg-gradient-to-r from-blue-500/5 via-purple-500/5 to-blue-500/5 backdrop-blur-sm border border-white/5 rounded-xl"
       >
-        <p className="text-gray-200" style={{fontSize: "12px"}}>
+        <p className="text-gray-200" style={{ fontSize: "12px" }}>
           {welcomeScreenData.welcomeSectionVersionBottom}
         </p>
       </motion.div>
     </div>
   );
-}
+}

src/hooks/useAnalytics.ts

@@ -1,9 +1,8 @@
 import { useState, useEffect } from 'react';
 import { fetchAnalytics } from '../lib/motherduck/queries';
-import { initializeConnection } from '../lib/motherduck/connectionManager';
+import { initializeConnection } from '../lib/motherduck/analyticsConnectionManager';
 import type { Analytics, LoadingStageId } from '../types/analytics';
 import type { DataLimit } from '../components/onboarding/DataSelectionStep';
-import { fetchSentimentTrends } from '../lib/motherduck/queries/sentimentTrends';
 
 export interface LoadingStage {
   id: LoadingStageId;
@@ -102,6 +101,16 @@ export function useAnalytics(
     groqStatus: groqToken ? 'Initializing GROQ...' : 'GROQ token not provided',
   }));
 
+  const updateQueryStats = (query: string) => {
+    setResult((prev) => ({
+      ...prev,
+      queryStats: {
+        count: prev.queryStats.count + 1,
+        lastQuery: query,
+      },
+    }));
+  };
+
   const updateLoadingStage = (
     stageId: LoadingStageId,
     status: LoadingStage['status'],
@@ -119,7 +128,6 @@ export function useAnalytics(
   };
 
   const handleError = (error: unknown, failedStage: LoadingStageId) => {
-    console.error('Analytics error:', error);
     const errorMessage =
       error instanceof Error
         ? error.message
@@ -141,6 +149,21 @@ export function useAnalytics(
     }));
   };
 
+  const handleProgress = (_stage: string, progress: number, currentQuery?: string) => {
+    if (currentQuery) {
+      updateQueryStats(currentQuery);
+    }
+
+    if (progress <= 20) {
+      updateLoadingStage(LOADING_STAGES.DATA, 'loading', progress * 2);
+    } else if (progress <= 60) {
+      updateLoadingStage(LOADING_STAGES.DATA, 'loading', 80);
+      updateLoadingStage(LOADING_STAGES.PROCESSING, 'loading', (progress - 20) * 2.5);
+    } else {
+      updateLoadingStage(LOADING_STAGES.VISUALIZATION, 'loading', (progress - 60) * 2.5);
+    }
+  };
+
   useEffect(() => {
     let isSubscribed = true;
 
@@ -164,29 +187,22 @@ export function useAnalytics(
         await initializeConnection();
         if (!isSubscribed) return;
         updateLoadingStage(LOADING_STAGES.CONNECTION, 'complete', 100);
-
-        updateLoadingStage(LOADING_STAGES.DATA, 'loading', 0);
-        const [analyticsData, sentimentTrends] = await Promise.all([
-          fetchAnalytics(database, tableName, limit),
-          fetchSentimentTrends(database, tableName, limit),
-        ]);
 
-        if (!isSubscribed) return;
-        updateLoadingStage(LOADING_STAGES.DATA, 'complete', 100);
+        updateLoadingStage(LOADING_STAGES.DATA, 'loading', 5);
+        setResult(prev => ({
+          ...prev,
+          queryStats: { count: 0, lastQuery: '' }
+        }));
 
-        updateLoadingStage(LOADING_STAGES.PROCESSING, 'loading', 50);
+        const [analyticsData] = await Promise.all([
+          fetchAnalytics(database, tableName, limit, handleProgress),
+        ]);
+        updateLoadingStage(LOADING_STAGES.DATA, 'complete', 100);
+        if (!isSubscribed) return;
         const processedData: Analytics = {
           ...analyticsData,
-          sentimentTrends,
         };
 
-        if (!isSubscribed) return;
-        updateLoadingStage(LOADING_STAGES.PROCESSING, 'complete', 100);
-
-        updateLoadingStage(LOADING_STAGES.VISUALIZATION, 'loading', 50);
-        if (!isSubscribed) return;
-        updateLoadingStage(LOADING_STAGES.VISUALIZATION, 'complete', 100);
-
         if (isSubscribed) {
           setResult((prev) => ({
             ...prev,

src/hooks/useGroqInsights.ts

@@ -7,7 +7,8 @@ export function useGroqInsights(
   substack?: string,
   analyticsData?: any,
   groqToken?: string,
-  model?: string
+  model?: string,
+  interests?: string,
 ) {
   const [insights, setInsights] = useState<string | null>(null);
   const [status, setStatus] = useState<StatusState>({
@@ -47,6 +48,7 @@ export function useGroqInsights(
           token: groqToken,
           stack,
           substack,
+          interests,
           positiveInsights: analyticsData.positiveInsights,
           negativeInsights: analyticsData.negativeInsights,
           emojiStats: analyticsData.textAnalysis.emojiStats,