ai_summer

ai_summer

Summary repository for AI Summer 2024

Stars: 59

Visit
 screenshot

AI Summer is a repository focused on providing workshops and resources for developing foundational skills in generative AI models and transformer models. The repository offers practical applications for inferencing and training, with a specific emphasis on understanding and utilizing advanced AI chat models like BingGPT. Participants are encouraged to engage in interactive programming environments, decide on projects to work on, and actively participate in discussions and breakout rooms. The workshops cover topics such as generative AI models, retrieval-augmented generation, building AI solutions, and fine-tuning models. The goal is to equip individuals with the necessary skills to work with AI technologies effectively and securely, both locally and in the cloud.

README:

Open In Colab

AI Summer

Summary repository for AI Summer 2024. Introduction to generative AI, with practical applications to inferencing and training

Presented by Vanderbilt Data Science Institute data scientists:

  • Dr. Jesse Spencer-Smith, Chief Data Scientist
  • Dr. Charreau Bell, Senior Data Scientist
  • Myranda Shirk, Senior Data Scientist
  • Umang Chaudhry, Data Scientist
  • Dr. Abigail Petulante, DSI Postdoctoral Fellow
  • Dr. Joshua Su, DSI Postdoctoral Fellow

Overview

The objective of these workshops is to develop foundational skills in understanding, inferencing and training generative AI models and other transformer models.

Getting Ready for AI Summer

Learn or Improve Your Python

Practice your Python skills using the below documents. Choose either a Google Colab for interactive programming environment, or alternatively read through the Google Doc.

Getting Accounts

You’ll want to use the most advanced AI chat model that you can get access to. Microsoft just opened access to BingGPT through Bing Chat, which is based on an early version of GPT4, currently the most advanced AI chat model available to the public. You’ll need to install the Edge browser (https://www.microsoft.com › edge › download) and go to bing. com. Click on “Chat”.

Decide on a Project (recommended)

Think about any data you might want to bring to the workshop. Also begin thinking about any projects you might want to accomplish during our month. We’ll have office hours for you to work with us to get your first project off the ground!

Workshop Schedule

Session will run live from 9am-11am, with an office hour from 11am to noon (all times Central).

Week 1, 5/6 - 5/8: Introduction to Generative AI Models, Transformers, and Prompt Engineering

Monday: https://vanderbilt.zoom.us/rec/share/17V0GyZJoD9PPmenE2stE9AAPGgT0x8KQcEis-VHMn4n8GdsUrjbmu61Tb8r4iv3.FZdNC_03OR0tiDSI?startTime=1715004136000

Wednesday: https://vanderbilt.zoom.us/rec/share/jmUNeAlIF-vBLDWmFuV_HWlI8c944oM3YB94Q_9H36pgKtwTvQlzQAyGASEM3gaw.OKGh4pXYucHTLKAv

No class Friday (Vanderbilt Commencement)

Weeks 2, 5/13 - 5/17: Retrieval-Augmented Generation (RAG), Assistants, Agents, and Intro to Diffusion Models

Monday Part 1: https://vanderbilt.zoom.us/rec/share/DFJ_bDybROtlo0qUseTFRo8QvntugqIsAlSVud1WKoEVlnO17ksxpz7uPgGgYN1r.1o_oPGd_07POUKux?startTime=1715608503000

Monday Part 2: https://vanderbilt.zoom.us/rec/share/DFJ_bDybROtlo0qUseTFRo8QvntugqIsAlSVud1WKoEVlnO17ksxpz7uPgGgYN1r.1o_oPGd_07POUKux?startTime=1715613842000

Wednesday: https://vanderbilt.zoom.us/rec/share/D_asoLsFb3tHZ4AB6qHaODUJvnwNyGQA6bQsG4pSMqvI1zQ-cQkNXsxcuPi1wqYv.vcKQy17Jc8Y0xGbM?startTime=1715781305000

Friday: https://vanderbilt.zoom.us/rec/share/y8paBkSmKLcUJRnHtwo8ZYSmNWCCG8nX9mtxfaOWBq_HbGSdMyq_VLi_E6wQZG2l.09Mt6t9VgpsR_ruj?startTime=1715954101000

Week 3, 5/20 - 5/24: Building AI Solutions, Running AI Securely Locally or in the Cloud, Introduction to Training Models

Monday:

Wednesday: Recording: (https://vanderbilt.zoom.us/rec/share/fswTlpFMlqAVgxRDDBza920i9brAuxaSiteHpDNUwpm9YQzedJa5g_2oZSSr2Eq1.wF73yKYGD5eY3cyY?startTime=1716392393000)

Friday: Recording: (https://vanderbilt.zoom.us/rec/share/plozihJcLFBIfjPxQ8Bsv9IdqHh39qFinkVUChsYtuiuiGAc8O2TcvTEbTE5cAUW.3XYBPJfbdZJ1GzAS?startTime=1716558902000)

Week 4, 5/29 - 5/31: Fine-tuning Models, Mulit-modal Models, Applied AI

No class Monday (Memorial Day)

Wednesday:

Papers/Blogs discussed:

https://arxiv.org/pdf/2405.17247

https://proceedings.mlr.press/v139/radford21a/radford21a.pdf

https://openaccess.thecvf.com/content/CVPR2022/papers/Singh_FLAVA_A_Foundational_Language_and_Vision_Alignment_Model_CVPR_2022_paper.pdf

https://arxiv.org/pdf/2405.09818

https://sh-tsang.medium.com/review-flamingo-a-visual-language-model-for-few-shot-learning-ec477d47e7bf

https://arxiv.org/pdf/2304.10592

https://arxiv.org/pdf/2310.03744

https://huggingface.co/papers/2311.05437

https://arxiv.org/pdf/2311.05437

https://llava-vl.github.io/blog/2024-01-30-llava-next/

https://llava-vl.github.io/blog/2024-05-10-llava-next-stronger-llms/

https://llava-vl.github.io/blog/2024-04-30-llava-next-video/

https://arxiv.org/abs/2310.02239

https://wandb.ai/byyoung3/ml-news/reports/How-to-Fine-Tune-LLaVA-on-a-Custom-Dataset--Vmlldzo2NjUwNTc1

Breakout Rooms

How to Breakout

Remember we are all learning and exploring

  • Please share your video upon entering the room and unmute
  • Share your screens--someone volunteer to share their screen upon entering, and everyone be ready to share your screen to show what you’ve found
  • Make notes of what you’ve discussed in the Response Reports below
  • Everyone be ready to report out (random)
  • Make some friends
  • Breakout Rooms Worksheets

Report Documents

Google Docs has a limit of 100 people viewing/editing a document at one time.

Special Breakout Room Groups

Please be sure your display name is set in Zoom. If you are in one of the following special groups, please pre-pend your name with one of the following qualifiers.

  • Data Science for Social Good: DSSG
  • Center for AI in Protein Dynamics: Protein
  • If you are in a lab and would like your own breakout room: Labname (keep it short, please!)
  • If you are faculty and would like to be in a breakout room with other faculty: Faculty

For example, I might be DSSG-Jesse Spencer-Smith

Workshop Video Recordings

Video recordings of these workshops can be found on our YouTube channel AI Summer playlist

Looking for the code resources for Summer 2023? View the 2023 repo version here.

Course Resources

Other Resources

Compute Grants for Vanderbilt Faculty and Students

DGX A100 Compute Grant: https://forms.gle/2mGfEy9DB4JU2GpZ8

Python

Transformers

  • Natural Language Processing with Transformers by Lewis Tunstall, Leandro von Werra and Thomas Wolf. If you are affiliated with Vanderbilt University, you can access this pre-print book (and any book by O’Reilly) free by logging into O'Reilly Media using your Vanderbilt email address. Vanderbilt licenses all content from O’Reilly. The book covers Transformers for purposes beyond text.

Getting the Most out of this Course

To get the most out of this workshop:

  • Open Colab (workbook) notebooks and actively write code along with the instructor
  • Actively participate in discussions
  • Actively participate in breakout rooms
  • Work on homework assignments before coming to class
  • Relax your mind and ask questions

Accessing Bing Chat

  1. Open the Edge browser (yes, Edge) and navigate to www.bing.com
  2. Select "chat". A new window should open saying you need the new Bing.
  3. Select "Start chatting" at the bottom of this window. This should prompt you to sign in to a Microsoft account. Do not use an organizational/school email (such as Vanderbilt). Instead, select "No account? Create a new one" and create one with your personal email. Note: if you get stuck in the "use the new Bing" window, go back to Bing.com and select "Sign in" instead. Follow instructions for Step 3.

  1. Sign in to your personal Microsoft account.
  2. Select "chat" - it should work now!

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for ai_summer

Similar Open Source Tools

For similar tasks

For similar jobs