This website is no longer maintained. It is kept for historical purposes only.

CSE 527: Introduction to Computer Vision

Spring 2019

Location and time: Mon, Fri 1:00 - 2:20, Room: ENGINEERING 143


In this course students will gain knowledge of theory and practice in Computer Vision, and by the end will have demonstrable ability to implement a working solution for real-world problems in image and video analysis. Students will get hands-on experience in deriving the mathematical underpinnings as well as the programmatic implementation of classical vision problems such as image classification, object detection and tracking, segmentation, pose estimation, visual scene reasoning (e.g. captioning, questioning) and more. Students will utilize deep neural networks of many types (convolutional, residual, recurrent, variational, adversarial, etc.), learn how to use cloud1 GPU-enabled virtual machines, evaluate their implementations on standard vision datasets, and compare their results to the state-of-the-art work of computer vision laboratories worldwide. This is an intensive hands-on class with weekly theoretical and practical programming assignments, which will give students the confidence to tackle computer vision problems in the wild as professionals.

1 We will use the Google Cloud Platform, courtesy of Google Cloud Education 


Date # Topic Class Topic Slides Assignment
1/28 1-1 Fundamentals,
Introduction to computer vision, prerequisites review PDF  
2/1 1-2 Human vision, optics, light, color, image formation PDF Hello Vision World
2/4 2-1 Image processing,
Images as functions, Convolutions, Filters, Edges PDF  
2/8 2-2 Interest points and corners, Features PDF  
2/11 3-1 Shapes, Models, Image matching and alignment PDF Image processing
2/15 3-2 Detection,
Object detection fundamentals: Templates, Haar-like features, HoG PDF  
2/18 4-1 Object tracking: Mean-shift, Kalman filter PDF  
2/22 4-2 Segmentation: Thresholding, Clustering, Superpixels, Graph-cuts PDF Detection, Tracking, Segmentation  
2/25 5-1 Multi-view geometry,
Stereo & 3D
MVG Intro, Epipolar geometry, Stereo PDF  
3/1 5-2 Stereo reconstruction PDF  
3/4 6-1 Camera calibration, Structured Light, Structure-from-Motion 1 PDF  
3/8 6-2 Structure-from-Motion 2, Multi-View Stereo PDF  
3/11 7-1 Machine learning Introduction, Linear regression and classifiers, SVM 1 PDF SfM
3/15 7-2 SVM 2, Logistic regression, Softmax, Analysis of classifiers performance PDF  
3/18 - 3/24 Spring break smiley 
3/25 8-1 Mid-term Mid-term N/A  
3/29 8-2 Machine Learning ML matters: Loss, gradients, SGD, overfitting, regularization. PDF  
4/1 9-1

Deep Learning 0:
DTrees, ANNs,

Decision trees, ensembles, boosting, biological and artificial NNs, MLP. PDF Hello Machine Learning World
4/5 9-2 BackProp, Convolutional NNs, Conv-Pool, ILSVRC, ImageNet, LeNet, AlexNet, Activation maps PDF  
4/8 10-1 Class Cancelled smiley N/A  
4/12 10-2 Deep Learning 1:
Classification &
Classification and Detection PDF  
4/15 11-1 Continued: Region proposals, SSDs, multi-task losses, YOLO and R-CNNs N/A  
4/19 11-2 Passover - No class smiley N/A  
4/22 12-1 Deep Learning 2:
Vision & Language
Keras tutorial. Intro to Vision & Language N/A Deep object detection
4/26 12-2 Image captioning, RNNs N/A  
4/29 13-1

Attention, Visual Attention, Visual Question Answering

5/3 13-2

Deep Learning 4:

AE, Variational AE, Generative Adversarial Nets 1 N/A  
5/6 14-1 GANs 2, DCGAN, VAEGAN, CGAN & InfoGAN PDF

Image Captioning, Generation

5/10 14-2 Review and prep Class review, final prep    

Piazza, Emailing, Appointments, TAs

We will be using Piazza for class discussion. Rather than emailing questions to the teaching staff, I encourage you to post your questions on Piazza so everyone may benefit from the answers.

Our class page is

Our TAs for this semester: 

My office hours are MF 2:30-4PM (after class) in room #145 in New CS.

Recommended Textbooks and Online Material

Computer Vision: Models, Learning, and Inference 
by Dr Simon J. D. Prince 
Link: Amazon, Download:
My notes: A good reference, but a bit outdated. Contains a good primer on relevant maths.
Computer Vision 
by Richard Szeliski 
Link: Amazon, Download:
My notes: Many algorithms, math underpinning and citations, but little intuition. Pre- deep learning.

Feature Extraction and Image Processing for Computer Vision, Third Edition 
by Mark Nixon 
Link: Amazon
My notes: Has all the elementary components of vision, but not much more.

Hands-On Machine Learning with Scikit-Learn and TensorFlow 
by Aurélien Géron 
Link: Amazon
My notes: Adequate coverage of scikit-learn and an old version of TensorFlow (0.9~1.0).
Deep Learning with Keras 
by Antonio Gulli, Sujit Pal
Link: Amazon

Deep Learning with Python
by Francois Chollet
Link: Amazon


  • Solid programming experience required: In this course you will program complete computer vision systems, from reading dataset files and processing pixels through coding an optimization problem. You should be comfortable picking up a new API (with some help). We will be working strictly in Python3 (OpenCV, NumPy, SciKit, TensorFlow/Keras, Jupyter). If you are uncomfortable in Python - please contact me.
  • Mathematics -- relevant knowledge required: This course relies on an established knowledge in computer science oriented mathematics. You are assumed to have taken undergraduate-level courses in the following: Calculus, Linear Algebra, Probability Theory, and Algorithms.
  • Machine learning experience very useful (but not required): Coding projects will usually have a machine learning component to them. It would be beneficial if you are familiar with basic ML pipelines (feature design, training, testing) and concepts (supervision, classification, regression, clustering, confusion matrices, precision-recall, ROC, AUC, F1 score, loss/gain functions, etc.)

If you have no solid programming2 background or no solid understanding of linear algebra3 and algorithms4you should not take this course.

If you do not have any experience in machine learning:

2 "solid programming" means you are capable of writing, documenting, running, debugging and analyzing the complexity of a program in Python given a non-trivial task at hand.
3 "solid linear algebra" means you have full grasp of concepts such as vectors, matrices, bases, operations, normalization, dot and cross products, factorizations, etc. and are able to derive equations by hand.
4 "solid algorithms" means you have full grasp of elementary data structures (lists, sets, vectors, trees, stacks, queue, hash tables), sorting, graph theory, combinatorics, dynamic programming and complexity analysis.


In this class grading will be given according to:

  • Attendance: Record of attendance will be kept in class. More than 5 absences (~20%), without just and confirmed cause5, will award a failing grade in the entire course. Grade component: 10%.
  • Attendance: Record of attendance until March 15 will make up for 5% of the final grade. Class attendance is not required through the end of the semester.
  • Assignments: Students are expected to complete all assignments. Missing assignments will award a failing grade for them. More than 3 missing assignments throughout the semester will award a failing grade in the entire course. Grade component: 35%.
    • Late submission: Assignments will be accepted only up to 48 hours past deadline. Each hour beyond the deadline will account for a 1.5% penalty decrease in grade. 
  • Mid-term: Students are expected to complete the mid-term. Missing the mid-term will not award a failing grade for the entire course. Grade component: 25%.
  • Final: Students are expected to complete the final to graduate from the class with a passing grade. Failing or missing the final might still award a passing grade in the entire course, but it's extremely unlikely. Grade component: 35%.

5 Confer with the university policies regarding absences:

Similar Classes

The following are classes of similar nature in which you may find useful information:

Class Policy Statements

Disruptive Behavior Zero-Tolerance Policy:

We will have absolutely NO abusive, demeaning, derogatory, offensive, defamatory, slanderous or rude communication in or out of the classroom, or any of the online channels of communication. There will be zero tolerance towards such behavior, and if such behavior will be observed it will lead to immediate reporting.

Stony Brook University expects students to maintain standards of personal integrity that are in harmony with the educational goals of the institution; to observe national, state, and local laws and University regulations; and to respect the rights, privileges, and property of other people.

Confer with the university policy on disruptive behavior:

Disability Support Services (DSS) Statement:

If you have a physical, psychological, medical or learning disability that may impact your course work, please contact Student Accessibility Support Center, ECC (Educational Communications Center) Building, Room 128, (631)632-6748. They will determine with you what accommodations, if any, are necessary and appropriate. All information and documentation is confidential.

Students who require assistance during emergency evacuation are encouraged to discuss their needs with their professors and Student Accessibility Support Center. For procedures and information go to the following website:

Academic Integrity Statement:

Each student must pursue his or her academic goals honestly and be personally accountable for all submitted work. Representing another person's work as your own is always wrong. Faculty is required to report any suspected instances of academic dishonesty to the Academic Judiciary. Faculty in the Health Sciences Center (School of Health Technology & Management, Nursing, Social Welfare, Dental Medicine) and School of Medicine are required to follow their school-specific procedures. For more comprehensive information on academic integrity, including categories of academic dishonesty please refer to the academic judiciary website at

Critical Incident Management Statement:

Stony Brook University expects students to respect the rights, privileges, and property of other people. Faculty are required to report to the Office of University Community Standards any disruptive behavior that interrupts their ability to teach, compromises the safety of the learning environment, or inhibits students' ability to learn. Faculty in the HSC Schools and the School of Medicine are required to follow their school-specific procedures. Further information about most academic matters can be found in the Undergraduate Bulletin, the Undergraduate Class Schedule, and the Faculty-Employee Handbook.

* Header image: