CIS 489/689 -- Computer Vision

An introduction to the analysis of images and video in order to recognize, reconstruct, model, and otherwise infer static and dynamic properties of objects in the three-dimensional world. We will study the geometry of image formation; basic concepts in image processing such as smoothing, edge and feature detection, color, and texture; segmentation; shape representation including deformable templates; stereo vision; motion estimation and tracking; techniques for 3-D reconstruction; and probabilistic approaches to recognition and classification.

Schedule

Mondays, Wednesdays, and Fridays from 1:25 pm to 2:15 pm in Smith 219

Prerequisites

This is a course for advanced undergraduates and graduate students.

We will be using Matlab to do programming assignments. No prior knowledge of Matlab is assumed, but experience with similar programming languages like C/C++ or Java is a must.

Familiarity with basic concepts in linear algebra (e.g., matrices, solving systems of equations), multivariable calculus (e.g., partial derivatives), probability, and statistics (e.g., covariance, outliers) is essential in order to follow the material.

Grading

60%	4 assignments worth 15% each. These will consist of programming in Matlab. Each homework will be due exactly 2 weeks after being assigned.
15%	Midterm exam
25%	Final exam

Students registered for the class at the graduate level will have an extra section to complete on every assignment, some extra readings, and more questions to answer on the midterm and final in order to earn full credit.

Programming assignments will be graded on the basis of correctness, efficiency, and originality. This is not a programming class, so coding style is not critical. However, if you want partial credit for something that doesn't quite work, it had better be well-commented and easy to follow. Grades will be given on assignments and tests in 0.1 point increments, rounded up (with 1 point = 1% of the total points possible in the course).

For the overall course grade, a preliminary absolute mark will be assigned to each student based on the percentage of the total possible points they earn according to the standard formula: A = 90-100, B = 80-90, C = 70-80, etc., with +'s and -'s given for the upper and lower third of each range, respectively. Based on the distribution of preliminary grades for all students (i.e., "the curve"), the instructor may increase these grades monotonically to calculate final grades. This means that your final grade can't be lower than your preliminary grade, and your final grade won't be higher than that of anyone who had a higher preliminary grade.

Academic

policies

Students can discuss problems with one another in general terms, but must work independently on writing and coding assignments. This also applies to online and printed resources: you may consult them as references (as long as you cite them), but the words and source code you turn in must be yours alone. The instructor reserves the right to use software to screen submitted code for similarity to other students' work and code available on the web. If it is determined that plagiarism has occurred on a homework or test, everyone involved will get a 0 for that assignment or test. A second occurrence will result in a failing grade for the course. The University's policies on academic dishonesty are set forth in the student code of conduct here.

The two exams will be closed book (i.e., no reference materials allowed). You are responsible for all material covered up to the day of the exam, both from the assigned readings (everything in the Readings column of the Schedule below, except things that are Matlab-related) and in lectures (excluding guest lectures).

Homework is due by midnight of the deadline day. As stated above, an on-time homework is worth a maximum of 15% of your total grade. You may turn in homework up to 3 days late, but for every calendar day that passes your grade on that homework goes down by 10% from what you would have gotten if it had been on time. If, for example, a homework is due on Friday at midnight, then anytime Saturday is one day late (officially starting at 12:01 am, but "slightly late" submissions will not be penalized), anytime Sunday is two days late, and so on. Supposing you turn it in on Monday morning, then the homework is 3 days late and you will be penalized 3 x 10% = 30% for lateness. In this case, if you would have earned 10.5 out of 15 points for an on-time assignment, 30% of 10.5 is subtracted, and you actually only receive 7.4 points.

Late days policy: You have 3 "late days" that can be used at any time to negate lateness penalties. This means you can turn in one assignment 3 days late, or 3 assignments 1 day late, without penalty. Just send an e-mail to the TA or instructor stating that you are using them at the time of submission. Once committed, they cannot be shifted to a later homework.

Homework submission

Assignment submissions should consist of a directory containing all code (your Matlab .m files) and output data (e.g., plots, images, movies, etc.--but do NOT send back provided input data!), and a simple web page index.html that explains what you've done and links to all of this in an organized way. More detailed instructions on function naming, calling conventions, etc. will be provided for each homework.

Every student will be given an ID code for grading purposes. The directory you submit for each assignment should be compressed after giving it a specific name based on your ID code. If, for example, you are a student named Jane Smith with ID code 0483 and you are turning in homework #5, then your directory will be called 5_0483. This directory must be compressed by tar'ing and gzip'ing it into a file called 5_0483.tar.gz or just zip'ing it into 5_0483.zip. The resulting file should be turned in by uploading it on the homework submission page (use the button at the top of this page).

Assignment and test grades (plus any comments) will be e-mailed to you when they are done. You will be prompted to give an e-mail address the first time you go to the submission page.

Computer labs

Evans 133 has ~16 Sun UltraSparcs; the room is restricted access with your UD ID and open around the clock (use the southeast door to the building after hours). I have no idea how many other classes will also be using this lab. For remote access, the names of these machines are listed here.

If you don't already have an ECE Research computer account or ECE Academic (ACAD or ECL) account, follow this link to sign up for an ACAD account on one of these machines:

http://www.eecis.udel.edu/NewAccount/

Grad students with ECE Research accounts automatically get an ACAD with the same login/password. For more information on accounts, etc., see the FAQ:

http://www.eecis.udel.edu/documentation/FAQ.html

Smith 040 (the classroom in the basement) has ~25 fairly up-to-date Windows XP machines. This room is open until midnight and publicly accessible when not reserved for classes, and therefore often full.

top

Readings

Note: You can get Adobe Acrobat Reader for reading PDF files here

Textbook (required)	v	Computer Vision – A Modern Approach D. Forsyth & J. Ponce [FP] Prentice Hall, 2002 Web site: www.cs.berkeley.edu/~daf/book.html Extra chapter 1: An Introduction to Probability Extra chapter 2: Tracking with Non-Linear Dynamic Models
Supplemental readings	v	Multiple View Geometry in Computer Vision R. Hartley & A. Zisserman [HZ] Cambridge University Press, 2000 Web site: www.robots.ox.ac.uk/~vgg/hzbook.html Chapter 1: 1-1.4 (pp. 3-24) (skip 1.2.3), 1.7.1-1.7.2 (pp. 28-31) Chapter 2: 2-2.2.1 (pp. 45-48) Chapter 3: 3.1-3.1.1 (pp. 71-73), 3.4.4 (pp. 91-93) Chapter 5: 5-5.3.1 (pp. 139-155), 5.3.4-5.3.5 (pp. 157-161) Chapter 8: 8-8.3, 8.6-8.6.1 (full chapter from book website; skip 8.2.2) Chapter 10: 10-10.2 (pp. 262-265) Chapter 11: 11-11.2 Chapter 17: 17.2 (skip 17.2.1)
	v	Computer Vision D. Ballard & C. Brown [BB] Prentice Hall, 1982 Chapters in PDF form
	v	Machine Vision R. Jain, R. Kasturi, and B. Schunck [JKS] McGraw-Hill, 1995 Moments Connected components Template matching
	v	Robot Vision B. Horn [H] McGraw-Hill, 1992 Convolution (Chapter 6.1-6.2)
	v	Introductory Techniques for 3-D Computer Vision E. Trucco and A. Verri [TV] Prentice-Hall, 1998 Optical flow (Chapter 8.1-8.2.1, 8.3-8.4.1)
	v	"Single View Metrology" A. Criminisi Proceedings of the 7th International Conference on Computer Vision Kerkyra, Greece, pp. 434-442, 1999 Journal version of "Single View Metrology" (more detailed explanations, more examples) International Journal of Computer Vision, 2001

top

Handouts

Matlab

Where to run

Evans 133: Matlab can be run locally on or remotely from these machines, and also on the CIS composer machines (strauss and mahler)

Smith 040: Matlab is accessible on the Windows machines here through the Start menu

If you want a version of Matlab that does not require being connected to the UD network while you work, you can buy a student edition for Linux, Windows, or MacOS from www.mathworks.com/store for $99; the Image Processing toolbox costs an additional $59.

Tutorials

Matlab links

For this class...

Please write your code so that you load class-provided input data from a different directory--e.g., for an image called test.ppm that you are reading in for homework #3, use the relative path of "../../Data/3/test.ppm". This will make the required non-inclusion of input data in your submission work.

Homeworks

Make sure that your index.html file for each homework includes your name, ID code, the homework number, and the date. If you are unfamiliar with HTML, there is a quick introduction here. Start with the "Minimal HTML Document" there and simply add sections corresponding to the different parts of each homework, putting in links where appropriate to your code, images, and results.

Instructions

Assigned

Due

1: HW 1: Single View Metrology

2: HW 2: Image Mosaicing

3: HW 3: Background Subtraction

4: HW 4: Classification

February 27

March 29

April 16

April 30

March 15

April 12

April 28

May 14

top

Schedule

Note: The blue squares in the "#" column below indicate Mondays.

The "Readings" complement and expand the lectures with which they are listed. I recommend studying

them before attending the lecture.

Certain mathematical symbols may display incorrectly in the Powerpoint files for the lectures without the

TexPoint add-in installed.

#	Date	Topic	Readings	Notes	Homework	Lecture
1	Feb. 11	Introduction				[ppt] [pdf]
2	Feb. 13	Matlab tutorial	some Matlab primer	Matlab scripts from lecture		[ppt] [pdf] [html]
3	Feb. 16	Geometry	BB: Appendix 1	2-D transformations, homogeneous coordinates		[ppt] [html]
4	Feb. 18	Geometry	FP: 2-2.1	3-D transformations, coordinate systems		[ppt] [html]
5	Feb. 20	Projective geometry	FP: 1-1.1, 2.2, HZ: 5-5.1	Projections, pinhole cameras, camera calibration matrix		[ppt] [html]
6	Feb. 23 Register/Add deadline	Estimation	FP: 3-3.1.1, HZ: 3.1-3.1.1, 3.4.4	Least-squares solution of linear systems (e.g., homogeneous)		[ppt] [html]
7	Feb. 25	Projective geometry	HZ: 1-1.3 (skip 1.2.3, 1.3.1), 2-2.2.1	Homogeneous lines, planes; lines, planes at infinity; rectification		[ppt] [html]
8	Feb. 27	Single view metrology	HZ: 1.7.1-1.7.2 (affine rectification), Criminisi: "Single View Metrology" (journal version)	Cross-ratio, measuring plane-to-plane distances	HW #1	[ppt] [html]
9	Mar. 1	Single view metrology		Reference distances, planar homologies, texture mapping		[ppt] [html]
10	Mar. 3	Images	JKS: Moments, connected components, template matching	Image arithmetic, thresholding, histograms, SSD		[ppt] [html]
	Mar. 5	NO CLASS
	Mar. 8	NO CLASS
11	Mar. 10	Filtering	FP: 7-7.2, 7.5-7.7	Convolution, kernels		[ppt] [html]
12	Mar. 12	Filtering	FP: 8-8.3.1, 9.2-9.2.1	Image gradient, basic edge detection		[ppt] [html]
13	Mar. 15	Edge detection	FP: 8.3.2	Canny edge detection	HW #1 due	[ppt] [html]
14	Mar. 17	Midterm review				[ppt] [html]
15	Mar. 19	MIDTERM
	Mar. 22	NO CLASS -- SPRING BREAK
	Mar. 24
	Mar. 26
16	Mar. 29	HW #2 background		Homography estimation, bilinear interpolation, image compositing	HW #2	[ppt] [html]
17	Mar. 31	Robust estimation	FP: 8.3.3, 15.5-15.6	Feature detection, RANSAC		[ppt] [html]
18	Apr. 2	Texture	FP: 7.3	Fourier transform		[ppt] [html]
19	Apr. 5	Texture	FP: 9-9.1, 9.2.2-9.4	Low-, high-, band-pass filtering, image pyramids, Gabor filters		[ppt] [html]
20	Apr. 7 Passover	Segmentation	FP: 14-14.3	Gestalt grouping, bottom-up and top-down approaches		[ppt] [html]
21	Apr. 9 Good Friday; Withdrawal deadline	Segmentation	FP: 14.4	Background subtraction, Mahalanobis distance		[ppt] [html]
22	Apr. 12	Segmentation	FP: 14.5, 15.1	k-means clustering, Hough transform	HW #2 due	[ppt] [html]
23	Apr. 14	Tracking	FP: 17-17.2 & An Introduction to Probability	Tracking as probabilistic inference; measurement likelihood, dynamics		[ppt] [html]
24	Apr. 16	Tracking	FP: 17.4-17.4.1, "Tracking with Non-linear Dynamic Models", 2-2.3	Particle filtering	HW #3	[ppt] [html]
25	Apr. 19	Tracking		conclusion of last lecture
26	Apr. 21	Classification	FP: 22-22.1	Posterior densities assumed Gaussian; nonparametric density estimation		[ppt] [html]
27	Apr. 23	Classification	FP: 22.2-22.3	Principal components analysis, Fisher's linear discriminant; eigenfaces		[ppt] [html]
28	Apr. 26	Classification		conclusion of last lecture
29	Apr. 28	Classification		Linear discriminants	HW #3 due	[ppt] [html]
30	Apr. 30	Classification	FP: 22.4	Neural networks, face detection	HW #4	[ppt] [html]
31	May 3	Stereo	FP: 10-10.1.1, HZ: 8-8.3, 8.6-8.6.1 (skip 8.2.2)	Epipolar geometry, fundamental matrix		[ppt] [html]
32	May 5	Stereo	HZ: 10-10.2	Estimating the fundamental matrix: DLT, rank constraint, nonlinear minimization		[ppt] [html]
33	May 7 Honors Day	Stereo	FP: 11	Depth estimation from stereo		[ppt] [html]
34	May 10	Structure from motion	12-12.1.1, HZ: 11-11.2	Triangulation, stratified reconstruction: projective-to-affine upgrade		[ppt] [html]
35	May 12	Structure from motion	FP: 12.3 HZ: 17.2 (skip 17.2.1)	Affine cameras, affine factorization algorithm		[ppt] [html]
36	May 14	Radiometry	FP: 4-4.3 (skip 4.2.3)	Irradiance, radiance; foreshortening; BRDF: Lambertian vs. specular	HW #4 due	[ppt] [html]
37	May 17	Shape from shading	FP: 5-5.4	Photometric stereo		[ppt] [html]
38	May 19	Final exam review				[ppt] [html]
	Wednesday, May 26	FINAL EXAM	10:30 am-12:30 pm Smith 219

top

Links

Other computer vision & robotics course homepages

The Computer Vision Homepage

top