CDIO Meeting Umeå

Greetings to my musings and ramblings on Engineering and Computing Education Research (EER and CER).

This first installment is notes and impressions from the Nordic CDIO conference in Umeå on the 5th and 6th of October 2010.

Happy reading, Arnold.


Day 1:

Interesting presentations at the schools of Architecture and Design at Umeå University.
The highly interactive and student focused approaches adopted in Design and Architecture are very interesting as a contrast to the more traditional engineering course structure. Two thirds of the teaching in the design school was done by staff from industry, and the financial and time committment from industry was considerable. To sustain this model the school admitted only 60 students per year, and had a total of approximately 100 staff and 150 students. Student body was very international with 29 nationalities represented.

Day 2:

Lennart Nilsson, Umeå

Talked about the structure of their programmes and the processes they use to gather information and this is then used to extract key indicators. However, the value of the data fed into the process is the key issue, I think the choice of data and what we believe about these data needs to be more widely discussed. The unbalanced dependence on student perceptions of aspects of quality has serious implications for long term development of higher education.

A key idea presented was the idea of black box analysis of programmes.

This area bothers me considerably, and I wrote an article presented as a Keynote at the Australasian Computing Education conference ACE 2010 on this topic, "Does quality assurance enhance the quality of computing education?"

Teijo Lahtinen: Lahti University of Applied Sciences, Finland

The talk described the approach adopted there. They have a long history on P&PBL in the Faculty of Science and Tech. since 1990, but the real shift to PBL was in 2000. Engineering adopted project and problem based education more fully in 2008 and now there is a standardisation of approach underway. Professional courses are early on in the programmes, there are 7hp projects in each year, involving teamwork, and PBL cases for the whole year with the supporting theory given in parallel. Project groups are 6-10 students. Projects involve more structured work in the first two years and then projects with industry in the last two years. A very good diagram of the CDIO and other quality processes, looking at both curriculum and staff development in parallel was presented.

Project Specification was open, but clear. All the basic restrictions and requirements were given, Project was to build a Segway transport device operating on 24 volts.  There are clear teaching teams for the first and second year to support the project learning. Students engage with the projects and are assessed both individually in the courses and then in the groups using peer assessment, project related documents, decision memos, timesheets and technical tests.

Gdansk University of Technology, Silwia S

Large university, 26K students, 1200 teaching positions, 130 prof. 140 DS, 600 PhD

Engineering and technology focus, no humanities apparently. Presentation of the general profile of the University, a classic international research and teaching university in the European tradition. They are interested to join CDIO as a part of the move from a subject oriented to a more industry and student oriented type of education. European and national qualification frameworks are forcing changes, and therefore adopting CDIO as a model for educating the modern engineer. Introduction of the concepts in Materials Science and Engineering in the Faculty of Mechanical Engineering 2011/2012. Examinations are large scale and general at the end of each term and years. Presentation of the final year projects to build motorized vehicles.

Observation that the students were more highly motivated to do design and use laboratory time to develop a practical project. The challenge is to understand the nature of the learning that takes place. The projects can involve staff from several faculties, this is a huge opportunity. The next step for them seems to be a more systematic approach to programmes instead of courses, and a focus on more general outcomes and structures.

It seems that there are two types of challenge in this area, one is the continuous inprovement and quality model used to analyse and develop the programmes, the other is the empoerment of staff to meet new classroom educational challenges helping them to adapt to the new pedagogical and didactic demands that emerge from this type of shift in educational practice.

Linköping Presentations

Ulf Nilsson: Peer-review of final theses. (LUX project)

Outcomes based evaluation of education is the new national approach. Quality of the final theses is an important aspect of this.
Härnqvist: En akademisk fråga - ESO rapport, http://www.evaluat.lu.se/publ/EN.pdf
Nilsson Direk kvalitetssäkring, http://www.evaluat.lu.se/publ/DirektKvalitetssakringRapp255.pdf

A meta-review based on a selection of theses at random, three reviewers were used per programme. Nine theses for each programme were reviewed, and the resulting individual reviews were discussed. The total number of theses from which the 9 were selected was not disclosed. Descriptions of the pre-requisites for entering the final thesis work were prepared, and the outcomes for the programmes and the expected outcomes and evaluation guidelines were compiled and distributed to the reviewers.

Assessment criteria

1. Problem formulation, appropriateness and ability to delimit the problem
2. Knowledge base, identify prior work, bibliographic references evaluated in relation to hte area and relevance.
3. Process demonstration of ability to work in independent and systematic manner within the time limits. This could not be evalated from the reports. Demonstration of theoretical awareness and use of well established "scientific methods".
4. Results and conclusions, adequate conclusions, relevant results, analysis of relevance and limitations, as well as demonstrating ethical awareness.
5. Conclusions, motivated and reasoned conclusions.

Reviewers meeting at the end of the process with program management and faculty representative. Conducted by videoconferences.
Some programmes experienced disagreement on the philosophy of the programme,  Personalvetareprogrammet och Sjukgymnastprogrammet seemed to be the ones that had the greatest discussion.
Software support for the review process would have been useful, seems like easychair would be a useful tool.
Grades were not useful, descriptives were more useful. High level qualitative descriptives are needed, since this is still too costly.

So what is the outcome? There was significant variation, however, there were poorer theses.
An important question is to understand the processes and the shortcomings in this process. The placement of students in industry with industry supervisors raises issues.

Cost of the exercise was about 80kkr

CDIO 2.0 Standard

New rubric 0-5 instead of 0-4 in the older standard.
Application of the standards and evidence of compliance exampes in order to help in the self evaluation process.
0-5 scale is related to normal quality process descriptors in terms of implementation of process, and continuous improvement measures.

Observations:
  • The process seems more efficient.
  • Standard 1: Adoption of hte principle. What does this mean? This is the tricky question.
  • Standards 9 and 10: dealing with faculty competence and development depends on the organisation of the university, Typically in Sweden program managers do not have influence over staff. No process and resources perspective is documented. There is no mention of student involvement in programme management, which is a key cultural feature of Swedish/Nordic education.
  • Still rather overweight in planning.
  • Standards 4 and 5 must be both satisfied if 5 is to be achieved, however the development stages described in Standards 1-3 are not necessarily encapsulated in higher standards.
  • Some standards require external evaluation for level 5, but not others. It is not clear why. this is the case.
  • The inclusive nature of the levels of achievement in each standard area should be re-examined.
Questions:

How does this differ from more traditional TQM approaches?
Why would there be a role in certification given ABET and EUR-ACE frameworks?


IUA and ITU/E matricies

Introduce
Teach
Utilize
Examine

Matricies for both course and programme level.

Does not the term "teach" in the codes imply a teacher-centric conception of educational activities?
Used to formulate and analyse achievement of programme goals, in accordance with sections 1-4 of the CDIO syllabus.
Use a top-down and bottom-up approach.
How to fill in the matrix? Guidelines
  • refer to the programme and course goals.
  • relate learning activities to the goals of the course and the programme
The collection and administration of matricies is being automated and an interface to a central DB implemented.

How to apply the matricies to embedded learning situations where learning is emergent from the learning situation and activities still seems to me to be problematic.

Johan Malmqvist: Constructive Alignments for degree projects, intended learning outcomes, teaching and assessment.

Contribution of constructive alignment to thesis work, what can be gained from constructive alignment?

Considerations of objectives, instructional/learning activities and instructional design related to assessment often leads to teachers having an "aha" experience and experiencing greater satisfaction in their teaching. But, how can this be applied, and possibly achieved in the context of the Swedish degree thesis projects? So what is this project?
  • A large exam, notthing new is taught?
  • vehicle for deepening knowledge and gaining professional skills?
  • internship or test employment?
  • cheap consulting hours for industry partners?
  • opportunity to publish a research paper and gain merits to help an application for a PhD programme?
So all these perspectives need to be considered. Depiction was related to a diagram that was very reminiscent of the activity theory triangle. This could be considered in more depth. An overview of some recent projects demonstrating very significant variation in activities and potential learning outcomes.
  • at what level should learning objectives been stated? National?  University? Programme?
  • remember that project goals are not the same as learning goals and gains, the new QA system may focus more attention on this.
  • Teaching and assessment, partially by others external to the university.
The issue of maximum and minimum goals needs to be considered, Chalmers has adopted a maximal goals description approach related to the higher degree ordinance, however, I think that Uppsala has a minimal set of goals, and all MUST be achieved.
A list of about 8 goals related to the higher education ordinance were presented.
Should all of these goals apply for all projects?
Spirited discussion about the nature of reports in the future, and the potential need to document working practice and artifacts produced as a part of the final report in order to meet quality assurance requirements in the future.

It seems important to have a national initiative to agree on generic outcomes and to discuss how to evaluate these. Clearer guidelines and assessment practices will help to increase the uniformity of reports and the way in which they demonstrate learning gains. Perhaps students should be active in formulating personal goals with respect to their own perceptions of their strengths and weaknesses.

Comments

Popular posts from this blog

The Future of Discipline Based Education Research

Why are We Here?

Does Campus Education Have a Future?