CSEQ and CSXQ Survey Operations Closed in 2014

After 35 continuous years of operation (since 1979), CSEQ and CSXQ survey operation were closed after the spring 2014 administration. New orders are no longer being accepted. However, institutions and independent researchers may apply to license items from the CSEQ and CSXQ for local assessment or research purposes. Researchers may also purchase extracts of de-identified data for analysis. Contact us at 812-856-5825 or cseq@indiana.edu for details. Thank you for your participation and support!

More details ....

C. Robert Pace UCLA 1975

A Summary of Pace's Career

Drawn from his 1998 Higher Education Handbook Chapter
"Recollections and Reflections"

University of Minnesota

Pace’s prolific career began as a graduate student in psychology at the University of Minnesota in the fall of 1933. He quickly developed an interest in measurement and joined the University’s Committee on Educational Research. Under the direction of Dr. Alvin C. Eurich, the committee examined the latest developments in educational testing and curricula through its experimental General College, as well as the occupational and economic effects of higher education through surveys of Minnesota graduates.

Upon graduation Pace directed the “adult study” at Minnesota which posited that good teaching required a general knowledge of students, and explored the idea that students have unique needs for skills and information as they developed as young adults. Under Pace’s direction the study examined four broad areas—personal, home and family, vocational, and social and civic. The resulting questionnaire, Building the University of Tomorrow, was packaged within a 52 page booklet (about 1,000 questionnaire items) and was ultimately completed by nearly 70% of those invited to participate! Paces’ analyses of these results became They Went to College, which was published by the University of Minnesota Press in 1941.

American Council on Education

Pace’s next stop was a post-doctoral fellowship with the Commission on Teacher Education, located in the American Council on Education in Washington, D.C. Pace and Dr. Maurice Troyer met with a range of higher education institutions to organize workshops, coordinate communication among them, and assemble useful instruments for colleges to evaluate their work. Through this work Pace developed the philosophy that education is improved when “the responsibility for the improvement of programs and procedures remained within each institution.” Thus, Pace and Troyer did not evaluate schools, but rather worked with them on evaluation tasks of their own choosing. This experience resulted in the publication of, Evaluation in Teacher Education, which Pace co-authored with Troyer and was published by the American Council on Education in 1944. The Commission’s approach is a forebear of the current assessment movement.

Civilian Scientist with the US Navy

Pace rejoined his former Minnesota colleague, Dr. Alvin C. Eurich, as a civilian scientist with the Bureau of Naval Personnel. Pace collected the opinions of enlisted men toward various aspects of their training and overall naval experience. The Bureau surveyed personnel about their post-war plans, which at the end of the war was combined with a similar research program at the Army (led by Samuel Stouffer, Paul Lazarsfeld, and Louis Guttman) to estimate the number of veterans who would likely go to college under the auspices of the GI Bill.

Guttman was doing much in the area of scale theory, developing a theory of content rather than a theory of responses which was typical in classical test theory. Pace traveled to the Pentagon on numerous occasions to visit with Guttman and to learn the concepts behind this new theory of scale development. This introduction to Guttman’s criteria shaped how Pace thought of scale development and directed his survey design efforts in the years ahead.

Following the war, the Office of Naval Research guided how the Navy contracted with universities in the pursuit of research of interest to the Navy. Pace was assigned to a review committee to read and recommend proposals, serving alongside such notable figures as Rensis Likert, Margaret Mead, Ruth Benedict, and Erich Fromm. In its first meeting the committee approved a proposal which established the Survey Research Center at the University of Michigan, which went on to become one of the most productive and influential centers for social science research in the nation.

American University

Pace briefly spent time developing an alumni survey for American University using the knowledge gained from his time surveying University of Minnesota graduates and his more recent experiences studying the work of Guttman. Pace constructed scales that measured aspects of the adult experience, including political affairs, civic affairs, art, music, literature, science, and religion. A variety of activities, ranging from the commonplace to those involving much greater intensity and commitment levels, were explored for each topical area ultimately resulting in scales. This approach would become the hallmark of much of Pace’s work related to higher education. A good scale could be developed with fewer than a dozen items, making data collection for a particular domain feasible in a way not previously possible.

Syracuse University

Pace joined the Syracuse faculty in the fall of 1947 just as the university was embarking on a massive self-study and was in the process of developing a graduate program in higher education. The comprehensive self-study, led by Dr. Maurice Troyer, head of Syracuse’s Evaluation Services Center, kept Pace actively engaged in an array of assessment activities. Multiple committees examined various aspects of the campus culture and work of the university, while Pace helped them in their efforts to gather available information and to develop the questionnaires necessary to solicit additional data. Each committee prepared a report detailing its efforts and recommendations and Pace then worked to condense their efforts into a single report of the study.

At the conclusion of the self-study Pace was invited to spend a portion of his time working as Special Assistant to the Chancellor where he frequently filled an institutional researcher and consultant role in diverse policy and planning discussions for the University. When Maurice Troyer left the university a few years later, it was quickly established that Pace would take over as director of the Evaluation Services Center. In addition to teaching the evaluation courses for the higher education program, Pace sometimes taught history and curriculum courses. As Director of the Center Pace led another large scale alumni survey, a revision and expansion of what he had developed for American University.

In 1952, Pace became chairman of the newly established psychology department at Syracuse, marking a change in his service to the university. Psychologists had previously served in multiple departments of the university; however, Pace now oversaw their collective work and had to establish a department that appropriately balanced these disparate fields of study. Pace served as chair for nine years before asking for the chance to return to his teaching and research.

National Organizations

In addition his work on behalf of the university, Pace had long been actively involved in a variety of national organizations including the College Entrance Examination Board, the American Council on Education, the Social Science Research Council, the Ford Foundation’s Fund for the Advancement of Education, and the Carnegie Corporation. His time working with the American Council on Education’s Committee on Measurement and Evaluation saw the establishment of the Educational Testing Service and sponsorship of the Cooperative Study of Evaluation in General Education.

In 1951 the Ford Foundation established the Fund for the Advancement of Education, and Pace was actively involved in various aspects of this organizations work as well. Pace’s former colleague, Dr. Alvin C. Eurich, had been named vice president of the organization and Pace often had opportunity to discuss potential policies and programs with him though his opinions often differed from those held by the Fund. Pace also was a regular contributor to various projects of the Fund, developing questionnaires to support assessment of projects on such diverse topics as liberal arts education, study abroad programs, and possible non-intellectual factors related to success in college.

Discussions within the research advisory committee of the College Board, led Pace to explore variables beyond SAT and high school grades which might be used to predict college success. A sub-committee was formed to explore additional factors, including factors in the environment that might foster success. Building off Henry Murray’s theory of personality needs and environmental press, the sub-committee encouraged the College Board to develop a set of environmental characteristics similar to personal characteristics already in use in a questionnaire developed by George Stern, a psychology faculty at Syracuse. The resulting College Characteristics Index (CCI) was then developed and tested at a handful of colleges. An additional grant from the Carnegie Corporation supported revision of the CCI and additional testing. Pace and Stern soon discovered their interests in using the CCI instrument diverged, Stern was interested in student level analyses while Pace wanted to explore the factors that differentiated between institutions. Pace conducted further investigations of the CCI while serving as a fellow at the Center for Advanced Study in the Behavioral Sciences in the 1959-1960 academic year. This work resulted in a new instrument, the College Characteristics Analysis (CCA) which was also a diagnostic instrument of academic and student sub-cultures on a campus. The CCA was drawn in part from the CCI, but half of the instrument was new content.

University of California at Los Angeles

In the fall of 1961, Pace joined the faculty of the school of education at UCLA bringing the CCA with him, while Stern continued to work with the original CCI form they had co-authored. At UCLA Pace found that some of the CCI items were unable to discriminate between campus environments, so he produced yet another questionnaire the College and University Environment Scales (CUES) which better discriminated at the institution level. The next decade was spent largely investigating the college environments. In 1962 the CUES was published by the Educational Testing Service and the rights to study academic and student sub-cultures was officially transferred from Syracuse to UCLA. Grants from the U.S. Office of Education funded numerous examinations of the measurement properties of the CUES throughout the early 1960s, all establishing a pattern of eight institutional types which opened new avenues of research in higher education. Funds from the U.S. Office of Education in the mid-1960s also helped to establish the Center for the Study of Evaluation at UCLA under Pace’s leadership. In 1974, The Carnegie Commission on Higher Education, published Pace’s work in the new Center under the title, The Demise of Diversity? A Comparative Profile of Eight Types of Institutions. This work made clear that real differences existed between some types of institutions. While the U.S. Office of Education did not fund the publishing of Pace’s book The Demise of Diversity?, it did commission the Center to develop a notebook outlining measures that any college or university might use for its own assessment needs. This Higher Education Measurement and Evaluation KIT, was released in 1971 and widely distributed among institutions.

Pace’s professional accomplishments continued into the 1970s, including his publication of Education and Evangelism, a review of Protestant colleges he conducted at the request of Clark Kerr. He served as an issue editor for a New Directions in Higher Education report on Evaluating Learning and Teaching in 1974 and published Measuring Outcomes of College in 1979, a review of student outcomes and the influence of institutions on the collegiate experience.

College Student Experiences Questionnaire

Finally, Pace’s work in the 1970s led him to question the concept of the input-environment-output model for evaluation and the practice of ignoring student characteristics when assessing the influence of an environment on outcomes. This work placed an increased emphasis on context and led to what Pace described as a contextual model expressed as environment-experience-development. This thinking led to the concept of a student’s “quality of effort” and ultimately to the College Student Experiences Questionnaire (CSEQ) in 1979. Understanding the quality of effort put forth by a student could add considerably to our understanding of student learning and development. This emphasis also encouraged placing some degree of accountability on a student in terms of the level of success they might or might not achieve. In other words, the new instrument recognized that some degree of initiative on the part of the student might be needed to attain desirable college outcomes.

Again employing Guttman’s approach to scale development, Pace spent two years developing the CSEQ with funding from the Spencer Foundation. The instrument included items on student’s background along with their experience in a variety of activities related to student use of facilities and personal and interpersonal events that typically occur in college. The CSEQ instrument was revised again in 1983 and in 1990 before Pace retired from UCLA and transferred the project to Indiana University under the direction of George Kuh, and a similar community college oriented instrument the (CCSEQ) to the University of Memphis under the guidance of Patricia Murrell.