The current version of the course web site and syllabus is available at https://cscie92.dce.harvard.edu/spring2021/.  Please refer there for updated information.

Syllabus for

Harvard Extension School CSCI E-92 (formerly CSCI E-251)

Principles of Operating Systems (25900)

Spring 2021
Site last revised 1:58 AM 24-Nov-2020

Dr. James L. Frankel

After the upcoming Spring 2021 semester, the next time CSCI E-92 will be offered will be either the Spring 2022 or Fall 2022 semester.

Quick Links:

• Class Hours & Location
• Distance Learning Links including Video Streaming, Chat, and the Midterm Exam
  • During Class
  • After Class
  • Midterm Exam
• Prerequisites
• Brief abstract
• Overview
• Books/Course Bibliography
• Instructor
• Teaching Assistants
• Questions and Issues
• Piazza Wiki/Forum
• Say Hello!
• Student Locations
• Using git
• Grading
  • Problem Sets
  • Late Policy
  • Commented and Documented
  • Programming Languages
  • Using git
  • Midterm Exam
• Accessibility
• Plagiarizing
• Course Outline
• Approximate Schedule
• Software and Course Documents On-Line
  • Slides used in class
  • Questionnaire and Problem Sets
  • Online Software: GNU, Cygwin, etc.
  • Harvard University Information Technology
  • Papers Used in Class
  • C* Information
  • Section Home Page

Tuesdays 7:40-10:15 PM ET via web conference. Students may attend at the scheduled meeting time or watch recorded sessions on demand. The recorded sessions are available within 24 hours of the lecture.

Distance Learning Links including Video Streaming, Chat, and the Midterm Exam:

During Class:

Video Streaming:

Both section & class are are live streamed and also recorded. Students are encouraged to share their video feed and to ask questions verbally using their audio/video link in Zoom. The section & class live video stream is available through the class Canvas web site under Zoom.

Chat:

Questions can also be asked using the text Chat facility in Zoom during class meetings & during section meetings. Please use the chat feature available in Zoom rather than Canvas chat. We will *not* be monitoring Canvas chat.

After Class:

Videos of class and section are available on the course's Canvas web site under Course Videos.

Midterm Exam

Our midterm exam is a three hour long proctored exam that will be available on-line through Canvas with integrity protected by Proctorio. Students must start the exam within the 24-hour period that begins at the designated start time of our midterm exam and ends 24 hours later. Students will have three hours to complete the exam independent of when they start. More information will be available about the exam later in the semester.

The exam does not allow access to any books or notes, but relevant course slides will be made available. No electronic devices are allowed.

Prerequisites:

Knowledge of data structures and programming experience, such as is taught in CSCI E-22 (formerly CSCI E-119) (Data Structures), is required. Experience using UNIX/Linux or equivalent at the shell level and the system programming level is strongly recommended, but not required. Students must have sufficient experience to write programming projects of modest size and complexity in the C Programming Language that utilize a variety of data structures. This course does *not* teach programming.

Brief abstract:

This course examines the role of operating systems: process synchronization and scheduling; memory management including virtual memory, swapping, paging, and segmentation; file management; protection and security; input/output techniques, buffering, and resource allocation; deadlock detection and avoidance; system modeling; performance measurement and evaluation; and operating system case studies. An extensive lab project will be required of all students.

4 credits. Graduate credit.

Overview:

Computer Science E-92 is a detailed exploration of the internal algorithms and design of operating systems. Students are expected to be comfortable with designing, coding, and debugging programs of reasonable complexity while employing good programming style and structured techniques. In particular, knowledge of C, terminal and text file I/O, classes, and data structures is presumed. During this class, students will use some UNIX system calls to complete the assignments.

A significant portion of the class will involve the design and implementation of a major term project. The project will be developed by each student working alone. That project is the core of a new operating system implemented from scratch for an ARM processor. Initially, both the classroom lectures and the section meetings will be covering material important to the design and implementation of the final project. Later in the semester, advanced topics will be covered in class; however, both the class and sections will continue to support students as term projects progress. For the term project, students will continue working on and debugging their projects leading to their complete implementation and a final demonstration.

Because the course includes a required and significant term project involving both programming and hardware implementation, the assignments will be time-consuming; therefore, a significant time commitment to the course is necessary. Although the relevant experience of students in the class is usually quite diverse, depending on background, it is not unusual for students to spend 10-15 hours per week or more completing the readings and homework assignments. Although the computers are available more-or-less around the clock, occasionally they will suddenly become unavailable (this is known as a crash). As with all such events, they always seem to occur at the worst possible time. Plan your computer work so that it is complete in advance of the deadlines. Check in your code to the required class git repository frequently. You have now been forewarned!

Books/Course Bibliography:

All course books are available from the Harvard Coop and are available for on-line ordering. A link to course materials at the Coop is available for on-line purchasing. Keep in mind that Coop members receive a 10% discount. There are links available on Canvas to find the Library Reserves and, for some books, these include View online versions. In addition, all registered students will be eligible for library services (access, borrowing privileges, group study rooms) in FAS libraries, just like any other student in FAS. Although Harvard's physical libraries are closed during the COVID-19 pandemic, the libraries are open on-line. All registered students will continue to have access to Harvard Library on-line resources.

Textbook:

Modern Operating Systems, Fourth Edition; Andrew S. Tanenbaum and Herbert Bos; Prentice-Hall, 2015, 2008; ISBN-10 0-13-359162-X; ISBN-13 978-0-13-359162-0

Optional UNIX and POSIX Library Function Reference Books:

Advanced Programming in the UNIX Environment, Third Edition; W. Richard Stevens and Stephen A. Rago; Addison Wesley Professional, 2013; ISBN-10 0-13-343547-4; ISBN-13 978-0-13-343547-4

PThreads Programming: A POSIX Standard for Better Multiprocessing; Bradford Nichols, Dick Buttlar, Jacqueline Proulx Farrell; O'Reilly & Associates, Inc., 1996; ISBN-10 1-56592-115-1; ISBN-13 978-1-56592-115-3

Recommended C Language Reference Manual:

C: A Reference Manual, Fifth Edition; Samuel P. Harbison and Guy L. Steele, Jr.; Prentice Hall, 2002; ISBN-10 0-13-089592-X; ISBN-13 978-0-13-089592-9; Errata for the Third Edition from Sam Harbison; Our additional errata

There will also be other handouts & supplementary readings

Instructor:

Dr. James L. Frankel 

• President, Frankel and Associates, Incorporated
• Office hours:
  • Via phone and/or web conference
  • By appointment

• E-mail address:   Questions whose answers would be relevant to the whole class should be posed via Piazza. For some appropriate issues (grading questions, etc.), e-mail should be sent to the professor and also to both TAs.
• Phone: +1.617.401.7480

Teaching Assistants:

We have two Teaching Assistants (TAs) for this course. The TAs hold a weekly section meeting and office hours as described below. Attendance at the TAs' section meeting is strongly recommended. Course material will be covered in section -- either in toto or in more detail -- that time does not permit to be covered in class meetings. For example, this material includes use of git and GitHub; general approaches to solving the problem sets; overviews of algorithms, code snippets, and data structures. Also, the section meetings provide a venue in which it may be easier to ask more lengthy questions. When appropriate to send e-mail, please send e-mail to both TAs and to the course instructor.

TA	Section Meeting Time/Place	Office Hours Time/Place	E-mail Address/Phone
Stephen Benjamin  , Section Site	Tuesday, 6:30-7:30 PM ET, Via web conference	Thursday, 6:30-7:30 PM ET by appt. only, Via phone and/or web conference	E-mail: ; +1.617.453.8107 (11:00 AM - 9:00 PM ET). If there's no answer, please leave a message with your name and a call-back number. Questions whose answers would be relevant to the whole class should be posed via Piazza. When e-mail is appropriate (for grading questions, personal issues, etc.), e-mail should be sent to both TAs and also to the professor.
Mark Ford , Section Site	Tuesday, 6:30-7:30 PM ET, Via web conference	Monday, 6:30-7:30 PM ET by appt. only, Via phone and/or web conference	E-mail: ; +1.978.496.7213 (1:00 PM - 9:00 PM ET). If there's no answer, please leave a message with your name and a call-back number. Questions whose answers would be relevant to the whole class should be posed via Piazza. When e-mail is appropriate (for grading questions, personal issues, etc.), e-mail should be sent to both TAs and also to the professor.

Questions and Issues:

When posing questions or bringing up issues of a non-personal nature, please use the class Piazza Forum. Answers to questions posed on Piazza benefit the whole class and allow the course staff to answer questions once for all students. Questions that include code or other information that shouldn't be shared with other students should be sent via e-mail to all course staff at the same time in order to increase the probability of a rapid response.

Piazza Wiki/Forum:

A Piazza Wiki/Forum (on-line discussion list) for CSCI E-92 is set up at Harvard Extension School CSCI E-92 Piazza Forum.

Record a Say Hello! Video:

Using any tool of your choosing (perhaps a cell phone selfie or the camera on your laptop), please record and post a short video (maybe just one to three minutes in length) in Canvas Discussions as a reply to my Say Hello! discussion to introduce yourself to the class. With everyone being remote, anything we can do to create a community for our class would be great. Please tell us a little about yourself possibly including where you are located, your background, what you do when you're not taking classes, and your goals for this class.

Enter Your Location:

Please enter your location in Canvas.

Using git and GitHub:

When using "git" and GitHub, make sure to follow the information on using "git" and setting up your GitHub repository that is available on the section web site.

Grading:

Graduate credit students:

• 65% Programming assignments & problem sets
• 12% Midterm exam
• 23% Term project (required)

Problem Sets:

All problem sets and programming assignments are due at midnight Eastern Time on Sunday night (i.e., midnight between Sunday and Monday) unless otherwise stated in the assignment or in the syllabus. Unless otherwise stated, all programming assignment solutions must be written in the C Programming Language. All code must build, be tested, and run on cscie92.dce.harvard.edu or on the NXP/Freescale K70 Tower computer systems, as appropriate; be submitted using "git" on GitHub (or, in dire circumstances, via e-mail only if agreed to by the course staff); be well-written (clear coding style, modular structure, appropriately commented and documented in English); and tested (include any programs and/or shell scripts used in testing your solution as part of your submission). Remember, in addition to handing in all parts of the problem set solution or programming assignment program, sample runs of the program which demonstrate that the program works must be attached. In addition, each submission must include a makefile to build the assignment. The grade for programming assignments will include all of these attributes.

Of course, the solutions may be written and tested using any system of the student's choosing; however, when the solution is complete, it must be tested on the cscie92.dce.harvard.edu computer and pushed to the git code repository on GitHub. You may choose to develop under your own Unix/Linux system or under Cygwin under Windows, but testing and grading of your programming assignments will take place on the cscie92.dce.harvard.edu computer. To reiterate, we will be grading the solutions based on their behavior on the cscie92.dce.harvard.edu computer.

You can establish an account on cscie92.dce.harvard.edu by accessing the website at URL https://ac-web.dce.harvard.edu/ and clicking on "Reset Password." Please note that the username shown on this screen is the username you will use to login to our server. Our cscie95.dce.harvard.edu computer may be accessed for remote login using "ssh" over the Internet. Files may be transferred to these systems using "secure ftp" (SFTP). If you are using a Windows system, the SecureCRT and SecureFX programs are available from the Science Center at http://downloads.fas.harvard.edu/download; these programs implement "ssh" and "secure ftp," respectively. On Unix/Linux systems, the shell commands "ssh" and "sftp"/"scp" can be used for ssh and SFTP, respectively.

Separate documentation is available describing how to install and use git and GitHub on the section web site.

Some assignments may include Extra Credit programming problems. The Extra Credit programming problems can be completed to earn points that can increase the overall grade on the programming portion of your problem set; however, the grade on the programming portion of a problem set including extra credit will ever exceed the full credit possible grade on the programming portion. That is, the Extra Credit programming problem(s) can be used to make up for deficiencies in other programming portions of the problem set to allow a higher grade to be earned. Extra Credit points from one problem set are not transferrable and may not be used on any other problem sets.

Late Policy:

All problem sets except for Problem Set 0 may be submitted late for partial credit. A late homework will lose 5% of its original grade for each day it is late (e.g. an assignment handed in two and a half days late will receive its original grade multiplied by 0.85). Late assignments may be submitted via "git" and an e-mail message notifying the instructor and the teaching assistants should be sent immediately after the late assignment is submitted. In addition, each student is given five free late days that may be used freely during the semester. However, keep in mind that almost all of the assignments are built on the previous assignments; handing in one assignment late does not extend the due date for subsequent assignments. The scope and difficultly level of the assignments increases during the class; therefore, we recommend against using the five free late days early in the class.

After a programming assignment has been initially submitted, we will award additional partial credit for corrections made to that assignment. We encourage students to correct any errors found in their code and to make improvements and enhancements. This will improve your grade and, in many cases, will be required to allow the next phase of your compiler to function correctly. No additional partial credit will be awarded for Problem Set 0 or for book problems.

Commented and Documented:

In the "Grading: Problem Sets" section above, the phrase "commented and documented" is used; this paragraph will clarify the necessary comments and documentation that should be provided with all programs. First, there should be a description of the entire application. This should include the user interface (i.e., how a user interacts with the program) and an explanation of what the program does. This documentation may be in a separate file from the program itself. Second, there should be a description at the beginning of each file which outlines the contents of that file. Third, each routine, function, method, etc. must be preceded by a section describing: (1) the name of the routine, (2) the purpose/function of the routine, (3) the parameters to the routine (name, type, meaning), (4) the return value from the routine (type, meaning), and (5) any side-effects (including modifying global variables, performing I/O, modifying heap-based storage, etc.) that the routine may cause. Fourth, declarations of variables should be commented with their purpose. Fifth, blocks of code should be commented to describe the purpose of the code section. Sixth, any complex or difficult to understand code statements or fragments should be commented to clarify their behavior.

Programming Languages:

In addition to programming in a conventional language (the C Programming Language), students will learn how to write code in ARM/Thumb assembly language. These are the low-level languages used by ARM computers. All students are required to use Special Edition CodeWarrior for Microcontrollers, Version 10.6 or Kinetis Design Studio (KDS), Version 3.2.0 from NXP/Freescale. This software may be downloaded from the www.nxp.com web site, is free, and no license is required. The Special Edition CodeWarrior for Microcontrollers software runs only on Windows. The Kinetis Design Studio software runs on either Windows, Mac, or Linux (DEB or RPM).

Using git:

When using "git" and https://github.com/, make sure to follow the information on using "git" and setting up your repository that is available on the section web site. Create a named branch for each of your problem sets as follows: specify "problem-set-0" for Problem Set 0 (the course questionnaire, fix this program, and word count), specify "problem-set-1" for Problem Set 1, "problem-set-2" for Problem Set 2, etc., specify "project-proposal" for the Term Project Proposal, and specify "term-project" for the Term Project.

Midterm Exam:

See Distance Learning Links: Midterm Exam for more information for distance students.

Accessibility:

The Extension School is committed to providing an accessible academic community. The Accessibility Office offers a variety of accommodations and services to students with documented disabilities. Please visit https://www.extension.harvard.edu/resources-policies/resources/disability-services-accessibility for more information.

Plagiarizing:

All work should be the personal creation of the individual student. Students are free to consult with each other and to study together, but all problem set solutions, programming assignments, exams, and the final project must be the personal contribution of each individual student. More explicitly, whenever a concept is reduced to a detailed algorithm or a program, no collaboration is allowed. If a paper, assignment, exam, program, or final project contains any information, algorithms, program fragments or other intellectual property taken from another source, that source and material must be explicitly identified and credit given. If you have any questions about this policy, it is the student's responsibility to clarify whether their activity is considered plagiarism.

You are responsible for understanding Harvard Extension School policies on academic integrity (https://www.extension.harvard.edu/resources-policies/student-conduct/academic-integrity) and how to use sources responsibly. Not knowing the rules, misunderstanding the rules, running out of time, submitting the wrong draft, or being overwhelmed with multiple demands are not acceptable excuses. There are no excuses for failure to uphold academic integrity. To support your learning about academic citation rules, please visit the Harvard Extension School Tips to Avoid Plagiarism ((https://www.extension.harvard.edu/resources-policies/resources/tips-avoid-plagiarism), where you'll find links to the Harvard Guide to Using Sources and two free online 15-minute tutorials to test your knowledge of academic citation policy. The tutorials are anonymous open-learning tools.

Course Outline:

• Devices for which the operating system is responsible
• CPU
• Memory
• RAM
• ROM
• Disk
• Hard
• Floppy
• CD-ROM
• CD-R
• Optical
• Magneto-optical
• MMU
• Display
• Keyboard
• Mouse (or other pointing device)
• Network
• Printer
• Modem
• Tape
• Bus
• DMA
• ISA
• PCI & PCI-Express
• USB
• SCSI
• IEEE 1394 (FireWire)
• Bootstrap
• Abstractions provided by the operating system
• Process
• Address space/Virtual memory
• File
• Window
• Socket, channel
• Interact via programmable shell & utility programs
• Operating system structure
• Monolithic
• Layered
• Microkernel
• Virtual machines
• Client-Server
• Exokernel
• Processes & Threads
• Process state and status
• Fork and join
• Synchronization, semaphores, monitors, critical sections, mutual exclusion, concurrency control, transactions, serializability, atomicity, deadlock
• Scheduling
• Processor
• Job
• Resource allocation
• Fairness, starvation
• Memory Management
• Paging & Swapping
• Virtual memory
• Page replacement algorithms
• Segmentation
• Files & Input/Output Systems
• File systems
• Polling vs. interrupts
• Buffering
• Memory mapping
• Sharing
• Typed memory and messages
• Persistence
• Clocks
• Power management
• System Modeling
• Queuing models
• Stochastic models
• Performance Measurement and Evaluation
• Cache, paging, VM, OS measurements
• Pre-fetch and instruction execution times
• Effect of register windows
• Introduction to Parallel and Distributed Systems
• Networking
• True distribution
• Pitfalls: network partitioning, dependencies and reliability
• Programming models
• Message passing
• Data parallelism
• Gang scheduling
• Parallel I/O
• Synchronization within jobs
• Security & Protection
• Systems calls
• Authentication & authorization
• Reliability
• Availability
• Privacy
• Common attacks & hazards
• Crypto-systems
• Kerberos
• Capabilities
• Access control lists
• Communication
• Synchronization
• Threads
• pthreads
• Remote procedure call
• Distributed file systems
• Replication
• Mobile computing
• Other operating system case studies

Approximate Schedule:

Software and Course Documents On-Line:

Slides used in class

• Class Agenda.
• Review of the C Programming Language for E-92 in PDF.
• Devices, Abstractions Provided, OS Structure in PDF.
• Processes in PDF.
• Basic Electronics in PDF.
• Memory Management in PDF.
• Input-Output Systems in PDF.
• Numeric Encodings for Operating Systems in PDF.
• Serial Communication in PDF.
• Deadlocks in PDF.
• Security and Protection in PDF.
• Parallel Systems in PDF.
• ISO C/POSIX/Unix/Linux I/O & Fork Concepts in PDF.
• Virtualization in PDF.

The Course Questionnaire and Problem Sets

• The course questionnaire is available on-line. Please submit your completed questionnaire using "git" with the tag "problem-set-0" as soon as possible after our first class meeting.
  
  
• Problem Set 0 (the course questionnaire, fix-this-program.c & word-count.c)
• Problem Set 1 (textbook problems & simple shell including conversion of microseconds since the Unix epoch to printable form)
• Problem Set 2 (memory management: malloc & free, K70 LED flashing program, shell enhancements)
• Problem Set 3 (textbook problems & device independent I/O for LEDs, pushbuttons, and an in-memory file system, shell enhancements, must use UART serial input & output for all shell interactions)
• Problem Set 4 (K70 enhancements: faster clock, off-chip SDRAM, supervisor calls, serial I/O added to device independent I/O, also LCD display, A-to-D input, and touch sensors, unprivileged mode, shell enhancements)
• Term Project Proposal
• Problem Set 5 (FlexTimer for time and date, interrupt-driven serial I/O, user timer, shell enhancements)
• Problem Set 6 (SysTick for multiprocessing, processes)

Online Software: GNU, Cygwin, etc.

• Look here for information about the GNU Project and the Free Software Foundation.
• Look here for information about getting GNU Emacs for Windows 95/98/ME/NT/XP and 2000.
• Look here for information about getting the Cygwin Linux-like environment for Windows.
• Look here for GNU manuals online.
• Look here for a List of Linux distributions.
• Look here for information about getting mtools: A collection of utilities to access MS-DOS disks from Unix.

Harvard University Information Technology

• There are computers available for Extension student use at 53a Church Street and in the Science Center, Room B-14. The Science Center computer labs are open 24 hours a day. Apple, PC, and Linux workstations are available for student use.
• Software is available for free download from Harvard Information Technology
• Harvard University Information Technology:
  • Harvard Extension School, Computer Services.
  • Harvard University Information Technology Home Page.
  • New FAS Account Creation. Also accessible via this link.
  • How to opt-in to Google Apps for Harvard and open a Harvard Gmail account.

Online Papers Used in Class

• Here is an interesting discussion of the Mars Rover Pathfinder mission involving priority inversion.
• Here's the discussion of the crash of the Ariane 5 rocket in HTML format and in text format.
• For papers discussing the Internet Worm of November 1988, see note by Gene Spafford & Robert Gasch, General Accounting Office report, University of Lowell report, How to patch the holes, the court appeal, request for comments 1135, MIT-IEEE, MIT, seely, spaf CCR, spaf ESEC.
• Description on blog.CryptographyEngineering.com by Matthew Green of the OpenSSL Heartbleed bug.
• Description of Diffie-Hellman key exchange vulnerability by Alex Halderman and Nadia Heninger.
• Description of Ingenious Chip-and-Pin Card Hack by Andy Greenberg of WIRED.

C* Information

• The C* Slides are available here in PDF format: The C* Language.

Section Home Page

• Section's Home Page