CSE 527: Introduction to Computer Vision

Fall 2017

Location and time: Tue, Thu 10:00 - 11:20, Room: Old CS 2120


In this course students will gain knowledge of theory and practice in Computer Vision, and by the end will have demonstrable ability to implement a working solution for real-world problems in image and video analysis. Students will get hands-on experience in deriving the mathematical underpinnings as well as the programmatic implementation of classical vision problems such as image classification, object detection and tracking, pose estimation, Structure-from-Motion, localization and mapping and more. Students will additionally learn how to train a deep neural network, write a GPU-optimized algorithm, evaluate their implementations on standard vision datasets, and compare their results to the state-of-the-art work of computer vision laboratories worldwide. This is an intensive hands-on class with weekly theoretical and practical programming assignments, which will give students the confidence to tackle computer vision problems in the wild.


Date / Week Topic Assignment Given
Tue 8/29

Practical C++ Programming Workshop (if registered, otherwise - no class)

Thu 8/31 Intro / Admins, Lin.Alg. review, Vision Concepts, OpenCV  HW0: Hello Vision World
Tue 9/5 Labor day (no class) smileysmiley
Thu 9/7 Optics, Light, Color, Human Vision, Cameras  
Tue 9/12 2D low-level: Convolutions, Filters, Edges HW1: Filters
Thu 9/14 2D mid-level: Interest points, Features, Descriptors  
Tue 9/19 2D high-level: Shapes, Models, Matching  
Thu 9/21

Rosh Ha'shana (no class)

Online: Stitching: Image Registration, Homography, Blending


HW2: Features & Stitching

Tue 9/26 Object detection: BoW, Viola-Jones, PCA  
Thu 9/28 Tracking 1: Optical flow, AAM, Snakes  
Tue 10/3 Tracking 2: Mean-shift, Kalman Filters, Particle Filters HW3: Detection & Tracking
Thu 10/5 Segmentation 1: Clustering, Superpixels, Mean-shift  
Tue 10/10 Segmentation 2: Mixture Models, Graph-cut, CRF/MRF HW4: Segmentation
Thu 10/12 Mid-term  
Tue 10/17 Multi-view 1: Cameras, Epipolar Geometry, Calibration  
Thu 10/19 Multi-view 2: Stereo, Structured Light  
Tue 10/24 Multi-view 3: Structure-from-Motion HW5: SfM / S-L
Thu 10/26 Multi-view 4: Visual odometry, SLAM 1  
Tue 10/31 Multi-view 5: SLAM 2 HW6: Vis.Odo. / SLAM
Thu 11/2 Machine Learning: Concepts, Vision applications  
Tue 11/7 Deep Learning 1: ANNs, CNNs  
Thu 11/9 Deep Learning 2: R-CNNs, FCNs HW7: DNN 1
Tue 11/14 Deep Learning 3: Classification, Segmentation  
Thu 11/16 Deep Learning 4: Pose Estimation HW8: DNN 2
Tue 11/21 GPU 1: Parallelizing vision tasks  
Thu 11/23 Thanksgiving break (no class) smileysmileysmiley
Tue 11/28 GPU 2: Practicalities, CUDA, OpenCL HW9: GPU
Thu 11/30 Document Image Analysis  
Tue 12/5 Class review, final preparation   
Thu 12/7 Final  

Piazza, Emailing, Appointments, TAs

We will be using Piazza for class discussion. Rather than emailing questions to the teaching staff, I encourage you to post your questions on Piazza so everyone may benefit from the answers.

Our class page is https://piazza.com/class/j6vr5jgnh9d366

Our TAs for this semester:

Schedule an office hours meeting with me: https://calendar.google.com/calendar/selfsched?sstoken=UVAzYmtkdzB4OHV1fGRlZmF1bHR8MjM2MDlmM2QwMDIyMGRjN2JiOTViMDVkNDIzMWU5OGU

Recommended Textbooks and Online Material

  • Computer Vision: Models, Learning, and Inference 
    by Dr Simon J. D. Prince 
    Link: Amazon, Download: http://www.computervisionmodels.com/
    My notes: A good reference, but a bit outdated. Contains a good primer on relevant maths.

  • Computer Vision 
    by Richard Szeliski 
    Link: Amazon, Download: http://szeliski.org/Book/
    My notes: Many algorithms, math underpinning and citations, but little intuition. Pre- deep learning.

  • Feature Extraction and Image Processing for Computer Vision, Third Edition 
    by Mark Nixon 
    Link: Amazon
    My notes: Has all the elementary components of vision, but not much more.
  • Hands-On Machine Learning with Scikit-Learn and TensorFlow 
    by Aurélien Géron 
    Link: Amazon


  • Solid programming experience required: In this course you will program complete computer vision systems, from reading dataset files and processing pixels through coding a Gradient Descent algorithm. You should be able to pick up a new language and API (with some help). We will be working in C++ (OpenCV, Boost) or Python (OpenCV, NumPy, SciKit). If you are uncomfortable in both Python and C++ - please contact me.
  • Mathematics -- relevant knowledge required: This course relies on an established knowledge in computer science oriented mathematics. You are assumed to have taken undergraduate-level courses in the following: Calculus, Linear Algebra, Probability Theory, and Algorithms.
  • Machine learning experience very useful: Coding projects will usually have a machine learning component to them. You should be comfortable with basic ML pipelines (feature design, training, testing) and concepts (supervision, classification, regression, clustering, confusion matrices, precision-recall, ROC, AUC, F1 score, loss/gain functions, etc.)

If you have no solid programming* background or no solid understanding of linear algebra** and algorithms***, you should not take this course.

We're offering a free "C++ Programming Bootcamp" workshop in the first 3 days of the semester: http://hi.cs.stonybrook.edu/cpp-workshop-fall17
Students who attend the workshop will get a 5% bonus on their final grade.

If you do not have any experience in machine learning:

You must be comfortable with learning a sizable set of new tools and processes quickly, and putting them to work right away. In other words, you are expected to be a Hacker :)

If you have a concern regarding your prior knowledge - contact me.

* "solid programming" means you are capable of writing, documenting, running, debugging and analyzing the complexity of a program in C++ or Python given a non-trivial task at hand.

** "solid linear algebra" means you have full grasp of concepts such as vectors, matrices, bases, operations, normalization, dot and cross products, factorizations, etc. and are able to derive equations by hand.

*** "solid algorithms" means you have full grasp of elementary data structures (lists, sets, vectors, trees, stacks, queue, hash tables), sorting, graph theory, combinatorics, dynamic programming and complexity analysis.


In this class grading will be given according to:

  • Attendance: Record of attendance will be kept in class. More than 5 missed classes (~20%) will award a failing grade in the entire course. Grade component: 15%.
  • Assignments: Students are expected to complete all assignments. Missing assignments will award a failing grade for them. More than 3 missing assignments throughout the semester will award a failing grade in the entire course. Grade component: 40%.
  • Mid-term: Students are expected to complete the mid-term. Missing the mid-term will not award a failing grade for the entire course. Grade component: 15%.
  • Final: Students are expected to complete the final to graduate from the class with a passing grade. Failing or missing the final might still award a passing grade in the entire course, but it's extremely unlikely. Grade component: 30%.
    • Project option: There is an option to do a project instead of taking the final. Projects will be done in up-to 2 people teams, and will require a significant programming and documentation effort. Consider this option only if you have a conflict with other finals or have a crippling fear of exams. This will probably be much more work than studying for the final.
    • Project is mandatory for students who take CSE 527 towards their CSE 522 capstone project requirement. Please let me know if this is the case for you.
  • Programming workshop: If you attend the workshop (and pass its exam) at the beginning of the semester you get a 5% grade bonus.

Similar Classes

The following are classes of similar nature in which you may find useful information:


Disability Support Services (DSS) Statement:

If you have a physical, psychological, medical or learning disability that may impact your course work, please contact Disability Support Services, ECC (Educational Communications Center) Building, room 128, (631) 632-6748. They will determine with you what accommodations, if any, are necessary and appropriate. All information and documentation is confidential.

[In addition, this statement on emergency evacuation is often included, but not required: 
 Students who require assistance during emergency evacuation are encouraged to discuss their needs with their professors and Disability Support Services. For procedures and information go to the following website: http://www.stonybrook.edu/ehs/fire/disabilities ]

Academic Integrity Statement:

Each student must pursue his or her academic goals honestly and be personally accountable for all submitted work. Representing another person's work as your own is always wrong. Faculty are required to report any suspected instances of academic dishonesty to the Academic Judiciary. Faculty in the Health Sciences Center (School of Health Technology & Management, Nursing, Social Welfare, Dental Medicine) and School of Medicine are required to follow their school-specific procedures. For more comprehensive information on academic integrity, including categories of academic dishonesty, please refer to the academic judiciary website at http://www.stonybrook.edu/commcms/academic_integrity/index.html

Critical Incident Management Statement:

Stony Brook University expects students to respect the rights, privileges, and property of other people. Faculty are required to report to the Office of Judicial Affairs any disruptive behavior that interrupts their ability to teach, compromises the safety of the learning environment, or inhibits students' ability to learn. Faculty in the HSC Schools and the School of Medicine are required to follow their school-specific procedures.