CSE 527: Introduction to Computer Vision
Location and time: Tue, Thu 10:00 - 11:20, Room: Old CS 2120
In this course students will gain knowledge of theory and practice in Computer Vision, and by the end will have demonstrable ability to implement a working solution for real-world problems in image and video analysis. Students will get hands-on experience in deriving the mathematical underpinnings as well as the programmatic implementation of classical vision problems such as image classification, object detection and tracking, pose estimation, Structure-from-Motion, localization and mapping and more. Students will additionally learn how to train a deep neural network, write a GPU-optimized algorithm, evaluate their implementations on standard vision datasets, and compare their results to the state-of-the-art work of computer vision laboratories worldwide. This is an intensive hands-on class with weekly theoretical and practical programming assignments, which will give students the confidence to tackle computer vision problems in the wild.
|Date / Week||Topic||Assignment Given|
Practical C++ Programming Workshop (if registered, otherwise - no class)
|Thu 8/31||Intro / Admins, Lin.Alg. review, Vision Concepts, OpenCV||HW0: Hello Vision World|
|Tue 9/5||Labor day (no class)|
|Thu 9/7||Optics, Light, Color, Human Vision, Cameras|
|Tue 9/12||2D low-level: Convolutions, Filters, Edges||HW1: Filters|
|Thu 9/14||2D mid-level: Interest points, Features, Descriptors|
|Tue 9/19||2D high-level: Shapes, Models, Matching|
Rosh Ha'shana (no class)
Online: Stitching: Image Registration, Homography, Blending
HW2: Features & Stitching
|Tue 9/26||Object detection: BoW, Viola-Jones, PCA|
|Thu 9/28||Tracking 1: Optical flow, AAM, Snakes|
|Tue 10/3||Tracking 2: Mean-shift, Kalman Filters, Particle Filters||HW3: Detection & Tracking|
|Thu 10/5||Segmentation 1: Clustering, Superpixels, Mean-shift|
|Tue 10/10||Segmentation 2: Mixture Models, Graph-cut, CRF/MRF||HW4: Segmentation|
|Tue 10/17||Multi-view 1: Cameras, Epipolar Geometry, Calibration|
|Thu 10/19||Multi-view 2: Stereo, Structured Light|
|Tue 10/24||Multi-view 3: Structure-from-Motion||HW5: SfM / S-L|
|Thu 10/26||Multi-view 4: Visual odometry, SLAM 1|
|Tue 10/31||Multi-view 5: SLAM 2||HW6: Vis.Odo. / SLAM|
|Thu 11/2||Machine Learning: Concepts, Vision applications|
|Tue 11/7||Deep Learning 1: ANNs, CNNs|
|Thu 11/9||Deep Learning 2: R-CNNs, FCNs||HW7: DNN 1|
|Tue 11/14||Deep Learning 3: Classification, Segmentation|
|Thu 11/16||Deep Learning 4: Pose Estimation||HW8: DNN 2|
|Tue 11/21||GPU 1: Parallelizing vision tasks|
|Thu 11/23||Thanksgiving break (no class)|
|Tue 11/28||GPU 2: Practicalities, CUDA, OpenCL||HW9: GPU|
|Thu 11/30||Document Image Analysis|
|Tue 12/5||Class review, final preparation|
Piazza, Emailing, Appointments, TAs
We will be using Piazza for class discussion. Rather than emailing questions to the teaching staff, I encourage you to post your questions on Piazza so everyone may benefit from the answers.
Our class page is https://piazza.com/class/j6vr5jgnh9d366
Our TAs for this semester:
- Shahira Abousamra email@example.com
Schedule an office hours meeting with me: https://calendar.google.com/calendar/selfsched?sstoken=UVAzYmtkdzB4OHV1fGRlZmF1bHR8MjM2MDlmM2QwMDIyMGRjN2JiOTViMDVkNDIzMWU5OGU
Recommended Textbooks and Online Material
Computer Vision: Models, Learning, and Inference
by Dr Simon J. D. Prince
Link: Amazon, Download: http://www.computervisionmodels.com/
My notes: A good reference, but a bit outdated. Contains a good primer on relevant maths.
- Feature Extraction and Image Processing for Computer Vision, Third Edition
by Mark Nixon
My notes: Has all the elementary components of vision, but not much more.
Hands-On Machine Learning with Scikit-Learn and TensorFlow
by Aurélien Géron
- Solid programming experience required: In this course you will program complete computer vision systems, from reading dataset files and processing pixels through coding a Gradient Descent algorithm. You should be able to pick up a new language and API (with some help). We will be working in C++ (OpenCV, Boost) or Python (OpenCV, NumPy, SciKit). If you are uncomfortable in both Python and C++ - please contact me.
- Mathematics -- relevant knowledge required: This course relies on an established knowledge in computer science oriented mathematics. You are assumed to have taken undergraduate-level courses in the following: Calculus, Linear Algebra, Probability Theory, and Algorithms.
- Machine learning experience very useful: Coding projects will usually have a machine learning component to them. You should be comfortable with basic ML pipelines (feature design, training, testing) and concepts (supervision, classification, regression, clustering, confusion matrices, precision-recall, ROC, AUC, F1 score, loss/gain functions, etc.)
If you have no solid programming* background or no solid understanding of linear algebra** and algorithms***, you should not take this course.
We're offering a free "C++ Programming Bootcamp" workshop in the first 3 days of the semester: http://hi.cs.stonybrook.edu/cpp-workshop-fall17
Students who attend the workshop will get a 5% bonus on their final grade.
If you do not have any experience in machine learning:
- Take an online course, for example:
- Pick up a book: http://amzn.to/2uqRlxy, http://amzn.to/2t3vOY2
- Be prepared to do a lot of self-learning.
You must be comfortable with learning a sizable set of new tools and processes quickly, and putting them to work right away. In other words, you are expected to be a Hacker :)
If you have a concern regarding your prior knowledge - contact me.
* "solid programming" means you are capable of writing, documenting, running, debugging and analyzing the complexity of a program in C++ or Python given a non-trivial task at hand.
** "solid linear algebra" means you have full grasp of concepts such as vectors, matrices, bases, operations, normalization, dot and cross products, factorizations, etc. and are able to derive equations by hand.
*** "solid algorithms" means you have full grasp of elementary data structures (lists, sets, vectors, trees, stacks, queue, hash tables), sorting, graph theory, combinatorics, dynamic programming and complexity analysis.
In this class grading will be given according to:
- Attendance: Record of attendance will be kept in class. More than 5 missed classes (~20%) will award a failing grade in the entire course. Grade component: 15%.
- Assignments: Students are expected to complete all assignments. Missing assignments will award a failing grade for them. More than 3 missing assignments throughout the semester will award a failing grade in the entire course. Grade component: 40%.
- Mid-term: Students are expected to complete the mid-term. Missing the mid-term will not award a failing grade for the entire course. Grade component: 15%.
- Final: Students are expected to complete the final to graduate from the class with a passing grade. Failing or missing the final might still award a passing grade in the entire course, but it's extremely unlikely. Grade component: 30%.
- Project option: There is an option to do a project instead of taking the final. Projects will be done in up-to 2 people teams, and will require a significant programming and documentation effort. Consider this option only if you have a conflict with other finals or have a crippling fear of exams. This will probably be much more work than studying for the final.
- Project is mandatory for students who take CSE 527 towards their CSE 522 capstone project requirement. Please let me know if this is the case for you.
- Programming workshop: If you attend the workshop (and pass its exam) at the beginning of the semester you get a 5% grade bonus.
The following are classes of similar nature in which you may find useful information:
- Brown: https://cs.brown.edu/courses/csci1430/
- Georgia Tech: http://www.cc.gatech.edu/~hays/compvision/
- Udacity: https://www.udacity.com/course/introduction-to-computer-vision--ud810
- UNC: http://www.cs.unc.edu/~lazebnik/spring11/
- Stanford: http://vision.stanford.edu/teaching/cs131_fall1617/
- Cornell: http://www.via.cornell.edu/ece547/
- U Rochester: https://www.cs.rochester.edu/~cxu22/t/577F16/syllabus.pdf
- U of Toronto: http://www.cs.toronto.edu/~kyros/courses/2503/syllabus.html
- Columbia: http://www.cs.columbia.edu/~areiter/CS_Webpage/COMS4731.html
- Illinois: http://slazebni.cs.illinois.edu/spring16/
- Duke: https://www.cs.duke.edu/courses/fall12/compsci527/syllabus.html
- MIT: http://6.869.csail.mit.edu/fa15/
Disability Support Services (DSS) Statement:
If you have a physical, psychological, medical or learning disability that may impact your course work, please contact Disability Support Services, ECC (Educational Communications Center) Building, room 128, (631) 632-6748. They will determine with you what accommodations, if any, are necessary and appropriate. All information and documentation is confidential.
[In addition, this statement on emergency evacuation is often included, but not required:
Students who require assistance during emergency evacuation are encouraged to discuss their needs with their professors and Disability Support Services. For procedures and information go to the following website: http://www.stonybrook.edu/ehs/fire/disabilities ]
Academic Integrity Statement:
Each student must pursue his or her academic goals honestly and be personally accountable for all submitted work. Representing another person's work as your own is always wrong. Faculty are required to report any suspected instances of academic dishonesty to the Academic Judiciary. Faculty in the Health Sciences Center (School of Health Technology & Management, Nursing, Social Welfare, Dental Medicine) and School of Medicine are required to follow their school-specific procedures. For more comprehensive information on academic integrity, including categories of academic dishonesty, please refer to the academic judiciary website at http://www.stonybrook.edu/commcms/academic_integrity/index.html
Critical Incident Management Statement:
Stony Brook University expects students to respect the rights, privileges, and property of other people. Faculty are required to report to the Office of Judicial Affairs any disruptive behavior that interrupts their ability to teach, compromises the safety of the learning environment, or inhibits students' ability to learn. Faculty in the HSC Schools and the School of Medicine are required to follow their school-specific procedures.