Document 6547987

 CRA Taulbee Survey User Manual
Version 1.0, October 8, 2014
Contents
Section
Page
For Users of the Previous Taulbee Survey: What’s Changed?
Getting Started
Who participates?
Taulbee User IDs
Taulbee Passwords
The Taulbee Survey Home Page
Getting Ready for Entry
Data Gathering
Types of Data Requested by the Survey
Data Gathering pdf
General Instructions
Data Entry
General Entry Procedures
Unable to Log In
Data Validation and Error Messages
Tips, Tricks, and Common Questions
Printing Your Entries
Submitting the Survey
Benchmarking (available spring 2015)
Benchmarking Basics
Predefined Reports and Slide Shows
Creating a Comparison Group
2
3
3
3
3
3
4
6
6
6
7
10
10
10
10
13
16
16
17
17
For any questions, email Dr. Betsy Bizot, CRA Director of Statistics and Evaluation at bizot@cra.org or call
her at 202-266-2943.
CRA Taulbee Survey User Manual V1.0 - 1
For Users of the Previous Taulbee Survey
If you are new to Taulbee in fall of 2014 or later, you can skip this section.
If you have been a Taulbee user before, here is an overview of the major differences.
1.
User IDs. Previously, there was one password per department and it was mailed in hard
copy. Anyone who had that password could see any of that department’s data. Now, there
will be one user ID for each person who will be working on the survey or using the
benchmarking features, and user permissions are more nuanced. A user may be given
entry capability without benchmarking or vice versa, or may be allowed entry of non-salary
data without access to the salary data. Your User ID is your email address.
2. Data Validation. The new system will conduct more data validation checks, including
comparisons to values from the previous year. You may therefore see more error and
warning messages than you are used to seeing. Some messages may be for values that,
on review, you find are correct; you may make comments for the survey manager on those
and leave them as is.
3. Access to Previous Year’s Data. Historical data from 2010-2013 has been imported into
the system; earlier data from 2004-2009 may be imported later. You may view or print pdfs
of your department’s past year’s data. From the Taulbee home page choose
Surveys/Survey Menu, click the link to “review Inactive Surveys,” and then choose the
desired year.
4. Peer Group Comparison/Benchmarking. Beginning in spring 2015, the new system will
allow users to select a peer group of institutions and view their unit’s data in comparison
to the peer group. The peer group capability will have the following restrictions:
a. Only CRA member units that responded to the survey in the most current year can
get peer group reports. (Exception: If you are a CRA member who is not eligible for
Taulbee, e.g. an industry lab or non-North-American institution, then you may
request a report for a selected group of departments.)
b. Only units that provide salary data can get peer group salary reports.
c. Units that provide individual-level salary data will receive a more complete peer
group salary report than those that provide summary-level data.
d. At least initially, each academic unit will be limited to a single peer group.
CRA Taulbee Survey User Manual V1.0 - 2 Getting Started
Who Participates?
The CRA Taulbee Survey is open to any North American academic unit that grants a PhD in
Computer Science, Computer Engineering, or Information (when the information has a strong
computing component). CRA membership is not required to participate in Taulbee, and nonmember departments receive the same early access to Taulbee reports as members. However, the
peer group comparison / benchmarking features of Taulbee are available only to CRA members.
The primary point of contact with CRA is most often the department chair but may be a senior
administrative person in the department, school, or college.
Taulbee User IDs
Each person who enters data into the online Taulbee or uses the benchmarking features needs
his or her own user ID. The Taulbee contact point (department chair or authorized representative)
can request IDs for the desired individuals by sending their names and email addresses to Dr.
Betsy Bizot, CRA Director of Statistics and Evaluation, at bizot@cra.org or survey@cra.org. By
default, users can access all data for their institution, but user IDs may also be set up without
salary permission. The User ID will be the email address and should be for a single person; rolebased IDs such as chair@ may be used only if they belong to a single individual.
Taulbee Passwords
The password to your Taulbee user ID must be at least 8 characters long, have at least 3 of the 4
of lower-case letters, upper-case
letters, numbers, and symbols, and
be changed at least once a year.
The Taulbee Survey Home Page
The online CRA Taulbee Survey is at
https://taulbee.cra.org
The first page will ask you to log in
with your email address and
password. The first time you log in,
you will be asked to accept an End
CRA Taulbee Survey User Manual V1.0 - 3
User License Agreement. After that, when you log in, you will see the home page, which has the
Welcome message on it. Enter the Taulbee survey by clicking the Taulbee Survey link under
Upcoming Deadlines. You can also choose the Surveys / Survey Menu option from the menu bar,
then select Taulbee.
Getting Ready for Entry
There are two steps that prepare the Taulbee Survey for entry by your academic unit each year.
These steps are taken once for the entire unit; they do not have to be done by each individual
user.
When you choose the
Taulbee Survey link from
the home page, you will see
the Survey Menu, of which
a portion is shown here.
This menu gives access to
both preparatory steps.
(The Survey Menu continues
below the portion shown
here with links to the
sections for Masters
Degrees, Bachelors
Degrees, Faculty, etc.)
Preparatory Step 1. “Will
Complete” box. Before
anyone in an academic unit
can enter data, someone
must check the “We will
complete this survey” box.
This is usually done by the
primary Taulbee contact for
the unit but can be done by
anyone.
CRA Taulbee Survey User Manual V1.0 - 4 If a unit chooses not to participate in Taulbee, they can click the “We won’t complete this survey”
box, and they will receive no more messages or reminders that year.
Preparatory Step 2. Reporting Structure Questions. Section A of the survey contains four
Reporting Structure questions that determine whether some of the later sections will display for
your unit. Those questions are:
a.
b.
c.
d.
Does your academic unit have an undergraduate program in CS/CE/I?
Does your academic unit have a masters program in CS/CE/I?
Did your academic unit award any PhDs in CS/CE/I during the previous academic year?
Will you be reporting salaries this year and, if so, at the individual or summary level?
Once the “will complete” box has been checked and the Reporting Structure questions in Section A
have been answered, the remaining sections of the Taulbee Survey may be completed in any
order.
Use of the “will complete” box and Reporting Structure questions will help CRA staff keep better
track of surveys in progress.
Note that the progress bars in the Survey Menu are approximate. Because some questions in
some sections may not apply to your unit (e.g. there are questions about CS, CE, and Information
students but you have only a CS program), the progress bar may not be completely shaded even
when all questions have been answered.
CRA Taulbee Survey User Manual V1.0 - 5
Data Gathering
Types of Data Requested by the Survey
In addition to Section A, Reporting Structure, the Taulbee Survey sections are:
B. Academic Unit Basics (total number of students and credit hours taught, academic calendar)
C. PhD Degree Titles
D. PhD Degrees Awarded
E. PhD Enrollment
F. Other PhD Questions (new students, students passing qualifying exams, expected degrees)
G. Masters Degree Titles
H. Masters Degrees Awarded
I. Masters Enrollment
J. Other Masters Questions (new students, expected degrees)
K. Bachelors Degree Titles
L. Bachelors Degrees Awarded
M. Bachelors Degree Enrollment
N. Other Bachelors Questions (premajors, new students, expected degrees)
O. Faculty Positions
P. Tenure/Tenure-Track Faculty Numbers
Q. Non-Tenure-Track Faculty/Postdoc Numbers
R. New Faculty Appointments
S. Faculty Salaries (individual or summary)
T. Research Expenditures and Doctoral Student Support
CRA Taulbee Survey User Manual V1.0 - 6 Data Gathering pdf
Many departments find it useful to have a pdf of the Taulbee Survey for collecting data to be
entered into the online form. The pdf will generally be available before the online form is open and
will be downloadable by anyone from www.cra.org/taulbee
This is NOT a typable pdf that can be used for electronic information transfer; it is solely a tool for
your use if you wish.
The pdf can also be downloaded from within the Taulbee survey by selecting the Downloads menu
tab and then clicking on the document title.
General Instructions
Completeness. If you cannot provide information for certain questions, please use the
“Unavailable” categories. For counts of students or faculty, you can fill in "0" rather than leaving
the cell blank. However, for dollar amounts such as salaries and research expenditures, if the
information is not available please leave the cell blank rather than entering a zero.
Consistency. Make sure that gender totals match ethnicity totals and that answers to different
questions do not conflict. This is especially important if more than one staff person is filling out
the survey. Wherever possible, the survey software will flag inconsistencies at the time of entry.
Otherwise, CRA staff will likely contact you for clarification.
CS, CE, and I only. Only include data for computer science, computer engineering, and information
programs with significant computing technical content. Do not include any data for fields such as
electrical engineering. If the title of your program is “Computer Science” (CS) or “Computer
Engineering” (CE) or “Information Science” (I) or similar, count students in those categories. If your
program crosses disciplines within computing (e.g. Computer Science and Engineering or
Computer and Information Sciences), count students in the appropriate category (CS, CE, or I) as
well as you can determine. If your program has a multidisciplinary name such as “Electrical and
Computer Engineering” or “Math and Computer Science,” count students only if their training
(doctoral dissertation, master’s thesis or project, or required courses for undergraduate major)
falls clearly within computer science, computer engineering, or information with significant
computing technical content, and then pick the most appropriate category (CS, CE, or I). Consult
CRA staff if you have a question.
CRA Taulbee Survey User Manual V1.0 - 7
Defining and Counting Faculty. Faculty includes tenured and tenure-track professors, and nontenure-track researchers, postdoctorates, and teaching faculty (i.e., instructors and lecturers).
Note that this includes only research faculty; research staff are asked about in the Profiles
section (conducted every 3 years. Count faculty as follows::
•
•
•
In questions asking for salaries, consider only faculty who are full-time and whose salaries
are paid primarily (50% or more) by your academic unit.
In questions asking for full-time equivalent (FTE) positions, consider as fractional FTE those
faculty who are either part-time or who are full-time with appointments in two or more
academic units.
In questions asking for counts by gender and ethnicity, give a head count (not FTE) for all
faculty who are at full time and whose salaries are paid 50% or more by your academic
unit.
Salary Data. We understand that salary data are sensitive. Data will not be reported in a way
that reveals individual persons or institutions, nor will they be seen by anyone other than CRA
staff. Canadian respondents should report in Canadian dollars.
•
•
•
•
•
Report salaries effective January 1 of the current academic year; if you are completing the
survey in the fall, as usual, then these are salaries for the following January. If you do not
yet know those, report the current salaries.
Report in nine-month equivalent. If your institution pays on other than a 9-month basis,
convert to 9-month. Except: Canadians, report 12-month salaries.
Report total salary. If a person is paid 66% from your academic unit, report their full salary,
not just the 66% of it you cover.
In order to allow more accurate and useful statistics, we request and strongly prefer
individual salary data (without names). If you absolutely cannot provide individual data,
please participate by completing the salary summary questions. You do not need to do
both; if you provide individual salary data, the system will compute the summaries.
For full and associate professors, salary statistics are calculated for ranges of years in
rank. If you provide individual salaries, you will be asked for the year the person was
promoted to that rank. If you hired someone at a higher-level rank, please provide the year
that person was originally promoted to that rank if available. If not, report the year hired at
your institution. If you provide summary salaries rather than individual, you are asked to
group associate professors into two bands (in rank 0-7 years, and in rank 8 years or more)
and full professors in 3 bands (in rank 0-7, 8-15, and 16 or more).
CRA Taulbee Survey User Manual V1.0 - 8 Premajor Undergraduates. In questions asking about undergraduate-level premajors, consider
those students who have declared, but have not yet been officially admitted into the academic
unit. This is more formal at some institutions than at others. If this status is not meaningful for
you, leave the items blank.
Ethnicity. When counting students or faculty, count each person in one racial/ethnic category
only. Do not include nonresident aliens in any other ethnic category. The following breakdowns are
drawn from guidelines set forth by the US Department of Education:
•
•
•
•
•
•
•
•
American Indian or Alaskan Native, Non-Hispanic: A person having origins in any of the
original peoples of North America, South America, or Central America, and who maintains
cultural identification through tribal affiliation or community recognition, but does not
report Hispanic or Latino ethnicity.
Asian: A person having origins in any of the original peoples of the Far East, Southeast
Asia, or the Indian Subcontinent. This category includes people from Cambodia, China, India,
Japan, Korea, Malaysia, Pakistan, the Philippine Islands, Thailand, and Vietnam.
Black or African American, Non-Hispanic: A person having origins in any of the black racial
groups of Africa, and not reporting Hispanic or Latino ethnicity.
Native Hawaiian or other Pacific Islander. A person having origins in any of the original
peoples of Hawaii, Guam, Samoa, or other Pacific Islands.
White, Non-Hispanic: A person having origins in any of the original peoples of Europe, North
Africa, or the Middle East, but does not report Hispanic or Latino ethnicity.
More than one race reported, Non-Hispanic: A person who reports himself or herself as
having origins in more than one of the racial groups listed above, but does not report
Hispanic or Latino ethnicity.
Hispanic or Latino: A person of Cuban, Mexican, Puerto Rican, Central or South American, or
other Spanish culture or origin, regardless of race.
Nonresident Alien: A person who is not a citizen or a permanent resident of the United
States (or Canada, for Canadian academic units) who is in the country on a visa or
temporary basis, and who does not have the right to remain indefinitely.
CRA Taulbee Survey User Manual V1.0 - 9
Data Entry
General Data Entry Procedures
Logging In. See the Getting Started section on page 3 for information on logging in to the system
and going to the Taulbee survey.
Survey Sections. Each section of the survey is a separate page; see the list of sections on page
6. Click the section title from the Survey Menu to begin entry for that section.
If a section does not appear on the Survey Menu, there are two possible causes. First, your user ID
may not be authorized to see that section (applies only to salaries). Second, if the Reporting
Structure questions in Section A have not been answered, or not answered correctly for your unit,
then sections for the Bachelors degrees, Masters degrees, or Faculty salaries may not display.
Previous year’s data. In many sections of the survey, data from the previous year is displayed.
This is for your reference; previous year’s data cannot be edited from the current year’s survey. If
you believe the previous year to be in error, then contact CRA at survey@cra.org to discuss
making corrections.
WARNING: Risk of Data Loss! In any section, you may lose data if you use your browser back
button after entering information. Always use one of the Save buttons at the bottom of the page.
If you are entering a large amount of data on one page, we advise occasionally using the “Save
and Stay Here” button during the process.
Unable to Log In
Your User ID is your email address. If you have more than one email address, be sure you are
using the one with which you were given a Taulbee ID.
If you forget your password, on the Log In page you can click the button to reset it. This will send
you an email with instructions on creating a new password. Also use this the first time you log
into the system.
CRA Taulbee Survey User Manual V1.0 - 10 Data Validation and Error Messages
Types of Error Messages. When you save data from a page, data validation rules are applied.
There are three types of error messages:
1. Fatal errors: Must be fixed before the survey can be submitted. There are only a few of
these; they are either missing required data, or have combinations of data that are
logically impossible.
2. Serious errors: Entries fall outside some preset limit or fields are blank when we expect
them to have data. Most errors in the system are of this type. Limits can be of several
kinds:
a. A fixed number, e.g. a min or max value for a field.
b. A comparison for logical relationships with other values within the same survey,
e.g. number of degrees awarded compared to number of students enrolled.
c. A comparison for logical changes from previous year’s data. This may take two
forms. For values less than 20, there is usually a check for a change of no more
than 5; for those 20 or larger, the check is for a percentage change.
These limits are set to catch many types of misinterpretations or typos, but some entries
that trigger “serious errors” will in fact be correct, and reflect legitimate large changes or
unusual situations. They also sometimes happen when only part of the survey has been
completed. You will have the opportunity to enter a brief comment explaining the entry
that triggered the error message, e.g. “Number of premajors decreased to 0 because
premajor status discontinued.” The CRA staff survey manager will review these comments
and, if necessary, contact you for more information.
3. Warning Messages: Entries are not what was expected; confirm that you intended the
entry as typed. No other action required.
What to Do About an Error Message. When you save a page and get one or more error
messages (see example on the following page), your responses HAVE been saved. You can :
1.
Click on the error message or the Return to Section button to go back to section with the
error and make a correction.
2. Enter a comment for the survey manager by clicking the Edit button next to the message.
3. Click Return to Survey Menu or Continue to Next Section buttons to disregard the error for
now. Serious errors will need to be corrected or commented on before the survey can be
submitted, but they do not have to be resolved before you can continue.
CRA Taulbee Survey User Manual V1.0 - 11
CRA Taulbee Survey User Manual V1.0 - 12 Tips, Tricks, and Common Questions
Degree Titles Sections
The Degree Titles entries at each level (bachelor’s, master’s, PhD) can be copied from the previous
year by clicking the “duplicate” icon to the left of list of responses from last year. If you hover over
it before you click it, it will say “copy.”
PhDs Awarded Section
Each degree recipient must have entries for field of study, gender, ethnicity, employment type,
employment location, and primary specialty, None of these fields can be left blank. They may,
however, be set to “unknown.”
When you click “add row” and begin entry of the information for a degree recipient, as soon as you
make an entry for any one of these fields, a message will appear below the Add Row button
saying that the form cannot be submitted because of the blank fields. (Secondary specialty and
the two text entry fields for other specialties may be blank.) Until you have either completed all
fields for that row or deleted the row, the save buttons are grayed out and you cannot continue
with other sections of the survey. As you select a value for each field, the corresponding bullet
point in the message will disappear; when all have been entered, the save buttons will be active.
CRA Taulbee Survey User Manual V1.0 - 13
Gender x Ethnicity Crosstab Tables
In many questions about students and faculty, information is requested in crosstab form as
shown below. If possible, please provide the gender x ethnic breakdown for that group of
students or faculty, using the “Not Available” row and column to count individuals for whom
gender and/or residency status are not known. (If an individual is known to be a U.S. citizen or
permanent resident, but his/her race and ethnicity are not known, count him/her in row h,
Race/Ethnicity Unknown.) Totals will be computed automatically.
If you cannot provide the gender x ethnicity breakdown, but are able to provide separate gender
totals and ethnicity totals, then enter the gender totals in row j, Not Available, and ethnicity totals
in the Not Available column. When you do this, the automatic total computation will show double
the number it should. This will be corrected when the survey is stored; you don’t need to do
anything about it.
If you cannot provide any ethnicity or gender information but do have a total number of
individuals, enter that total number at the intersection of the Gender Not Available row and the
Ethnicity Not Available column.
CRA Taulbee Survey User Manual V1.0 - 14 Progress bars
Progress bars are approximate because not all questions apply to all academic units. Therefore,
the progress bar may not be completely shaded even if you have entered all of your data for a
section.
Arrow keys
You cannot use the arrow keys to move from field to field in the Taulbee survey pages. Tab or
shift-tab can be used to move to the next or previous field, or you can click on any editable field
on the page to jump to that field.
CRA Taulbee Survey User Manual V1.0 - 15
Printing Your Entries
At the bottom of the Survey Menu is this Completion Process box. The middle item allows you to
print a copy of the survey for review or for your records.
Submitting the Survey
Submit the survey by checking the “We have completed this survey!” box in the Survey Status box
at the top of the Survey Menu. If you have unresolved serious or fatal errors within the survey,
you will be given a list of the errors that must be resolved before the survey can be submitted.
Fatal errors must be
fixed. Serious errors
may be fixed, of
course, but if the
entry is correct, the
error may be
resolved for
submission by
entering a brief
explanatory
comment.
The “Cleaned &
Frozen” and “Under
Follow-Up” statuses
CRA Taulbee Survey User Manual V1.0 - 16 are used by the CRA survey manager.
Peer Group Comparison/Benchmarking
Beginning in spring 2015, the Taulbee Survey will allow users to select a peer group of institutions
and view their unit’s data in comparison to the peer group. The peer group capability, also referred
to as benchmarking, will have the following restrictions:
1.
Only CRA member units that responded to the survey in the most current year can get
peer group reports. (Exception: If you are a CRA member who is not eligible for Taulbee, e.g.
an industry lab or non-North-American institution, then you may request a report for a
selected group of departments.)
2. Only units that provide salary data can get peer group salary reports.
3. Units that provide individual-level salary data will receive a more in-depth peer group
salary report than those that provide summary-level data.
4. At least initially, each academic unit will be limited to a single peer group.
More information on Benchmarking will be added to this manual before it is made available in
spring 2015.
CRA Taulbee Survey User Manual V1.0 - 17