Submission and Assessment of Programming Assignments

How to submit programming assignments.

  1. Make sure that you have an executable copy of the generic submit script "submit.sh" in your ~/.bin directory. This script is available in LIB/scripts/. This script is the same for all assignments.
  2. Make sure you have a copy of the file "deliverables.sh" from the LIB assignment directory. The deliverables.sh script configures the generic submit.sh for the specific course and assignment. Each deliverables.sh is assignment-specific whereas submit.sh is an engine that is independent of a particular course and assignment.
  3. Make sure you are logged in to shell or quake and in your assignment directory.
  4. Enter the command "submit.sh" (or "submit.sh deliverables.sh") and observe the screen. Any messages should be attended, especially those appearing in a red font.
  5. Check your CS email. Be sure that you have received two messages from the course home:
    • The first message is sent as soon as your submission is received at the course home. This message acknowledges receipt.
    • The second message is sent after your submission has been unpacked and placed in the assignment directory in your portfolio. This message contains a copy of all of the files that were unpacked and placed in the assignment directory.
    Both messages are important. In particular, open and read the second message to be sure the contents accurately reflects your submitted files. If you do not receive both messages in a few seconds it usually means that an error or malfunction has occurred.

AutoCheck. Most assignments have an autocheck (aka robocheck) utility built in to the submission processing system. When autocheck is active, you should receive a 3rd email that contains the results of auto-check on the submitted assignment. This shuold give insights into how well your assignment will assess:

  1. It gives a (partial) assessment rubric for the assignment.
  2. It gives the results of building and testing your submission, using a battery of standard tests.
  3. Based on the results data, you can see how well your submission will fare when it is assessed with the standard tests.
  4. It would be highly advisable to correct any issues detected by auto-check and resubmit the assignment.
  5. Note that more tests may be added during assessment, so autocheck should not be considered definitive and definitely should not be relied on as a replacement for your own testing, which should be exhaustive. Even passing all robotests does not ensure your code will pass all assessment tests.
  6. Note that the grade assigned during assessment will be based on (possibly expanded) testing results as well as other factors such as adherence to our Code Standard, quality of software engineering, and adherence to the assignment specifications and requirements.
  7. AutoCheck should not be used as a substitute for your own build-revise-test process. Therefore the limit of 5 robochecks is placed on each assignment, with an additional 5 during the grace period.

log.txt. Each programming assignment will have a required log.txt as part of its deliverables. This file serves at least two purposes. The first is simple documentation of when you worked on the project. The second is to record a "test diary" in which your testing process is outlined and documented. Particular attention should be paid to errors that are uncovered during testing and how they were diagnosed and corrected.

This should be an ascii text file in your project directory with the following header:

log.txt 
<date file created>
<your name>
<your CS username>
<course number and assignment name>

This file should document all work done by date and time, including all testing and test results.

The test results should be in a separate section of the log called "test diary" and should document how you plan to test and what the results are.

Your submitted log.txt will be used during assessment. We will always assess whether it meets the requirements. We may also use log.txt to ameliorate point losses for testing failures, depending on what is in your test diary.

Due dates, grace periods, and late submissions. Each programming assignment will have a due date announced in the course organizer. Submissions may be made at any time and as often as desired, as long as they are received before the due date. New submissions will just overwrite the previous submission.

Once the due date arrives, a 5-day grace period begins. (Typically, due dates are at the end of the weekend (Sunday) and the grace period extends through the following Friday. All days end at midnight eastern time.) During the grace period, submissions are date stamped (NOT time stamped), and each date stamped submission in the portfolio will "cost" 4% of the grade off the top. Note these subtleties:

  1. Once a date-stamped submission has been made, all subsequent submissions on the same date will overwrite the existing one, and hence they cannot be counted against the grade. So, once you have submitted on a given day, all other submissions on that day are "free".
  2. The number of days between submission is also not relevant to the grade, so it is sensible not to submit after the deadline until you are confident the project will receive a good assessment.
  3. Beware of midnight eastern time - the date changes at that instant. The best time to submit is in the morning, which leaves the rest of the day to resubmit without penalty.

Once the grace period ends, submissions will be marked LATE and will not be assessed. Auto-check does run on LATE submissions, so you can submit for feedback even though no credit is given for late assignments.

Assessment. Programming assignments will be assessed by a member of the instructional staff. First the instructor will run the auto-check script interactively, observing the action on screen and assembling a test report. The instructor will then read the test report, the source code, and other deliverables, and arrive at a grade for the assignment using a rubric designed for the assignment and the requirements stated in the assignment document.

Each assignment will have detailed assessment rubric modified from the following general guidelines:

 Assignment Assessment Rubric Guidelines - details may vary 
 Criterion   Approximate Percentage Points Range   
 Deliverables Received and Project Builds Cleanly     0 ... 25 
 Results of Testing   0 ... 25 
 Project Meets Requirements  0 ... 25 
 Software Engineering Considerations   -75 ... 25 

Software engineering considerations include readability, design, style, safety, and appropriate use of language and library features. Note that any choice you make is important and subject to evaluation. Thus every choice should be made thoughtfully and with a rationale that can be given for making that choice. (See Code Stds for more details.)

Once the assignment has been assessed using the rubric, a detailed grade report will be emailed to the student. Note that automated test results as well as the assessment report will be emailed to the student's CS email address.

More about submit scripts. The submission process requires two things: (1) a submission engine that does the processing - gathering up the files, encoding them, and emailing them to the appropriate location. And (2) a configuration file that gives the details on what files are to be submitted, what the project name is, and what course should get the submission.

The submission engine is the same for all assignments and is named "submit.sh". A copy of this file should be put in your .bin directory and made executable, effectively making "submit.sh" a command invokable anywhere in your login so that it will not require copying into various assignment directories.

The specifics for any given assignment are contained in the configuration file, required by submit.sh. The default name for the configuration file is "deliverables.sh". A submission configuration file will be supplied for each assignment.

Setting up submit.sh the first time. Here are details on operationalizing the submit.sh command:

  1. Get the submit script. Copy the file LIB/scripts/submit.sh into your .bin directory:

    cp LIB/scripts/submit.sh ~/.bin/

    Note: "LIB" has a specific definition for your particular course and is defined in the organizer for your course.

  2. Make submit.sh an executable command. Change permissions:

    chmod 700 ~/.bin/submit.sh

    and then either log out and back in or run the command "rehash" to operationalize the command.

  3. Get the assignment configuration file. Copy the file LIB/assign0/deliverables.sh into assign0 directory.

    cp LIB/assign0/deliverables.sh ~/[course_name]/assign0/

    (See Course Organizer for definition of "LIB".)

Once the submit.sh command is set up, it should work for all assignments in classes using the submit system. It is a good idea to update the script by re-copying at the beginning of each new semester, just in case there has been an upgrade, but the command setup will remain, all you need do is copy LIB/scripts/submit.sh into ~/.bin/.