|
|
Heads of University of Biological SciencesLondon Meeting STANDARDS & QUALITY: TEACHING AND LEARNING IN THE BIOLOGICAL SCIENCES Tuesday November 7, 2000 Royal Institute of British Architects 66 Portland place, London W1N 4AD Minutes Announcement: The next Heads of University Biological Sciences meeting will take place in Reading on the 18/19th April, 2001. The theme of the meeting will be the Impact of New Technology on Biology: E-Biology: Learning and Publishing. New themes such as bioinformatics are to be included. Contributions or ideas are welcomed.
Apologies: Mike Laugharne (QAA) was not able to attend as a speaker.
Professor David Weitzman (Higher Education Consultant) " A 7-year itch: QAA Subject Assessment & Review - A Septennial Learning Perspective"
What I would like to do is reflect on my experience over the years. The 7 year itch referred to in the title relates to the number of years that the QAA has been running. In my time I have led 84 subject reviews. Some lessons may be learnt by reflecting on the experience and by not just focussing on the Biosciences. There are useful experiences across the subjects that are worth sharing and may be taken forward into the "Academic Review", which is the title of the new methodology/system to replace the QAA.
To start off, I would like to voice my opinion that the term "Teaching Quality Assessment" (TQA) is quite an unfortunate term, but it is one that the QAA uses. It misrepresents what the assessment is about. The assessment is not solely focussed on teaching. Of course, teaching is involved in it, but it is only one part of it. The term TQA misses out on the equally important aspect of learning and the quality of learning. We should in fact focus more on learning. The term "TQA" ignores such things as curriculum design, student welfare and so on. The term "TQA" also promoted a lot of unnecessary anxiety in the lecturers. The alternative term "inspection" led by "inspectors" is not any better.
We have to accept the fact that the majority of lecturers are not professional teachers, but in fact amateur teachers. They may be professional biologists in their relevant fields, but they do not necessarily hold a teaching qualification. The QAA review provides the opportunity for lecturers to demonstrate professionalism in teaching. We should, therefore, have a positive attitude towards the QAA. It is an opportunity to demonstrate to the outside world that we are in fact professional in the way we go about teaching our subject and that we achieve high professional standards.
Teaching and the dissemination of knowledge can in a way be likened to the processes surrounding DNA. DNA can be compared to knowledge. The accurate replication of DNA represents teaching. The expression of DNA, in turn, represents the application of knowledge. Mutations or the evolution of DNA represents the change of given knowledge by new research and results in the field. We need a mechanism to monitor replication and mutation, and this mechanism is what the QAA reviews represent.
In trying to assess the quality of education, we need a working definition of "quality of teaching". What does "highest quality" mean. What does "high quality teaching" mean? These are terms often found in the mission statements of various institutions, but when pressed for an explanation, many members of these departments cannot explain what these terms really mean. There is a need for members of a department to explore what this means. The main part that needs to be completed in a self-assessment is entitled "The evaluation of the quality of education". As biologist you would, therefore, expect to have a definition of "quality" before you start to evaluate it. I (David Weitzman) have read at least 200 self- assessments in the course of leading reviews from various universities and I would like to share the information I have been able to gain from this.
The "Aims and Objectives" section of a self-assessment is one section that is crucial to get right. The overall and specific objectives need to be distinguished from each other. Very often the difference between an MSc and a BSc aims and objectives was simply that the MSc was somehow "deeper" or "more". In turn this implies that a BSc is "less". Clearer distinctions of undergraduate and postgraduate levels need to be expressed.
The 6 aspects of educational provision that we would expect to be covered in any type of assessment are as follows:
1. What are the courses offered and what do they contain? How are they designed and programmed? How do they fit into the declared purpose? How do they fit in with the aims and objectives (Show that they are fit for the purpose). 2. How (how well) do we deliver the courses? What do we expect the students to get out of them? How do we monitor what students have gained from them? How effective is that monitoring process? 3. Who are our student "customers"? How do they get on with the courses and how well do the do at the end and beyond? 4. What is the quality of support and what are the customer care systems? 5. What is the quality and the adequacy of resources in support of the provision? 6. How (how well) do we monitor the success and effectiveness of our operation and how (how well) do we introduce progressive improvement in what we do? Many of the Self Assessment Documents (SAD) that I have read and evaluated lacked evaluation. Most often they are largely descriptive and very procedural.
Strengths that were encountered were: 1. Range of courses offered 2. Impact of staff research on teaching and learning 3. Effect of sandwich teaching where it is offered 4. High quality teaching encountered 5. Good range of assessments (except for the weakness in the lack of feedback to students) 6. Good completion rates 7. High level of student achievement 8. High level of student support
Weaknesses that were encountered were: 1. The curriculum sections are very descriptive. There seems to be a difficulty in evaluating them. 2. Teaching, learning and assessment: the effectiveness of these points is not evaluated, only described. It is necessary to build the description into the evaluation of effectiveness. A description alone is not enough. 3. No overall strategy in TQA. No strategic approach to teaching. 4. Achievement is only expressed in narrow terms:
5. No strategy in student support and guidance.
6. Career's Advice
7. Learning Resources
The points discussed in this weaknesses section are things that are important to deal with. It is important to have an overall strategy. This process of critical evaluation is here to stay and will be carried forward into the new review process. I would like to highlight some of the points in the handbook of the new method. It very much focuses on the process of evaluation. The self -evaluation document is central to the review process. It is designed to make the institutions evaluate the quality of teaching offered. It is an opportunity to reflect on what we do, they way we do it etc. Academic reviewers will expect to see evidence of careful self-analysis. The self- assessment document is one of the most important documents in the review process and is one that is available to the reviewers before the review. The SAD needs to be evaluative and not solely descriptive. Far too many universities hand in descriptive evaluation rather than evaluative ones. This has repercussions for the universities. If a good evaluative and reflective document is produced it will lead to a "lighter touch" during the reviews. Conversely, if a poor document is produced, the reviews are led with a "heavier touch".
1. There is a word limit on the SAD, which makes self-evaluation more difficult. 2. An evaluative document was written initially, but the self-evaluative sections were removed by people above. The word limit just needs to be worked at and is not a real excuse. Self-evaluative sections seem to be removed because they are perceived as being self-critical. Self-evaluation and self -criticism does not however mean that just the faults need to be highlighted, but also the strengths. A department should be confident to state conditions as they are. If problems are identified, these should be mentioned and methods for resolving them should be suggested. This would then be perceived as a strength rather than a weakness, because the problem was addressed. The weakness would be perceived as a weakness if it were not identified. This in turn would lead to a heavier review.
1. The grading system was criticised. However, this grading method was based on votes by institutions. The new method to be introduced may change this grading system. 2. The TQA has been described as an artificial exercise. The departments put on a "show". However, the preparation for the review is a legitimate part of the review process and is a part that will hopefully carry on after the review. 3. The cost of the reviews has been criticised as being to high. However, those that complain about the cost are admitting indirectly that their departments were in bad state and needed a lot of financial input to rectify the situation. The cost of the review is generally considered a reasonable price to pay if the benefits that were intended are achieved.
Discussion:
David Bigmore (Queen Mary University of London): Would you define for me again what you mean by the term "reflective"? It is term that is popping up everywhere now, and I have encountered it in the nursing curriculum.
David Weitzman: Well, they use the term "reflective practice" a lot in nursing, don't they? When I use the term "reflective" in self-assessment, I mean nothing more sophisticated than standing back and thinking about what you are saying, and thinking about what you are doing, and thinking about the prompts that this has offered you and has invited you to respond to. To reflect on these points and not simply to think this is what they want and returned a descriptive list of what you do. What is the purpose of the things you undertake, how are these achieved? This is all I mean by reflection.
Chris Clayton (University of Birmingham) This is more a comment than a question on the lack of evaluation in self-assessment documents. I felt that although there is so much stress on evaluation, the final reports handed back by the QAA are themselves not evaluative but more descriptive.
Chris Skidmore: Everyone talks about the so-called League tables, which are created. These can have an effect on the institutions, for example the reputation of institutions, links with professional bodies and just the whole morale of the institutions. What does a score of 24 mean? Is this satisfactory? No, it must be excellent. Where is the cut off point between satisfactory and excellent? People start creating demarcation lines. It is interesting when you go back to 1993, when you had unsatisfactory, satisfactory and excellent, when it comes to the crunch, below a certain award score, it is still unsatisfactory. It is curious that we still think in satisfactory, unsatisfactory and excellent, albeit in a certain scale, which is what we now have.
David Weitzman: And the numbers are meaningless. If you have a score of 22, and you have lost two points, where you have lost two points can tell you a lot. The idea originally was that the scoring system would provide a profile, which would show where the strengths are and where the weaknesses lie. The loss of two points would show where there was scope for improvement. This is all lost, however, because what people do is add them up and all that information is lost.
David Skidmore: Reflecting on what David is saying I think the way with which we cope with assessment is somewhat of a cultural thing with us. Just think about the way in which we assess our students. We initially do create a wonderful profile with a lot of information about a student, but then we hide and lose all this information by averaging this information and concentrating only on the averages. Take a student that completely fails his independent project yet receives a good average mark because they are excellent at producing the required coursework and essays in exams. All this valuable information on this student is lost in the average mark that he is awarded at the end of his degree. This is not entirely satisfactory, yet it is a cultural thing.
Dr Simon van Heyningen (University of Edinburgh) "Benchmarking Biological Sciences"
To start of with it is important to mention that the benchmarking process in Biological Sciences is only at its beginning. The Biological Sciences Benchmarking Group has only had one meeting so far. The push for benchmarking originally came from the Dearing Report. The Dearing Report is very often the reason behind everything that is undertaken by the Quality Assurance Agency (QAA). The Dearing Report, item 10.60 states: ".We have debated whether it is important, or necessary, for UK higher education awards to have a meaning which is commonly accepted and recognised. One option would be for UK awards to go in the direction of a number of other countries, such as Japan and the USA. In these countries, there is no national attempt to have awards of common standards across institutions or subject, except in some professional areas. Instead, there is an informal ranking of institutions, with parents, students, academics and employers knowing, at least in broad terms, the relative status of each.". As you know, we have not gone along this route and decided to opt for common quality assurance procedures.
Item 10.66 of the Dearing Report states further: "We are attracted to the proposition that standards should be developed by the academic community itself through formal groupings from the main areas of study.In many cases, subject associations and professional bodies will play a role in developing benchmarks." The problem exists, however, that nobody is certain what the term "benchmark" actually really means. We are meant to develop this term ourselves. Item 10.67 of the Dearing Report gives us an idea of what they had in mind: "The benchmark information could be used by institutions as part of their programme approval process to set degree.standards. In addition, it should be used by external examiners to validate whether programmes are within the agreed standards for particular awards." This statement led to Dearing Recommendation 25: " We recommend to the Quality Assurance Agency that its early work should include: - to work with institutions to establish small, expert teams to provide benchmark information on standards, in particular threshold standards, operating within the framework of qualifications, and completing the task by 2000." The first and foremost comment to make about this recommendation is the fast, almost silly timetable. In my opinion, all these things are being done far too fast. Trying to complete this task by 2000 would leave no time for reflection and no time for institutions to take this on board. Dearing had in mind that some sort of levels should be laid down of a minimum requirement, a threshold, for a graduate in any particular subject. There was much discussion at the time about what this meant, whether this was useful at all, and what a "minimum requirement" really represents. If we think about "thresholds", as mentioned in the recommendation, we are talking about a level above which a student just scrapes a 3rd Class Degree. In principal that is what I think a benchmark is all about. When a student has a degree in X, he should know a minimal amount of things about X. Are we really interested in such a minimum threshold? We can imagine a situation where this might be of major relevance to employers. Dearing, for example, was more interested in the field of engineering. It is easy and useful in this field to decided what minimum information is required for graduates and even in some other fields of study. What is this minimum threshold for Biology, though? In the field of biology there aren't really any easily identifiable core bits that could be deemed relevant as minimum knowledge required by a student.
Let us consider next for whom the benchmarks are actually intended. Who will find them useful? To do this we need to look at what benchmarking should be used for. It is important to bear in mind that the working definition of benchmarks only concerns undergraduate Honours degrees.
Benchmarking could be used for:
Of these, point 4 is very useful. In addition, in practical terms, documents produced for points 2 (academic review) and 3 (external examining) will not be useful for point 6 (informing the public about the subject and its standard). One cannot possibly write documents that satisfy all these points. The general agreement therefore is that, for now, benchmarks should be written for other academics about the subject, even though this is not what they were initially intended for. Despite this settlement it was still not particularly clear exactly what the benchmarks are supposed to do. What level of student achievement are they dealing with? Who are they really intended for? How much detail is supposed to go into them? The result of this is that in practice most documents that have been written have dealt with the transferable skills gained and not on the factual content of subjects. Many of these transferable and generic skills are actually common to all graduates and not specific to graduates of a particular degree.
A large amount of benchmarks have already been prepared. Examples exist for subjects such as Chemistry, Law and History. Each subject has applied different approaches and produced very different benchmark statements. For example, chemistry's benchmarks are rigid and the document looks like more akin to a syllabus. In the case of history the benchmarking process was not so much based on factual content, but more on the general skills acquired by a student during their degree programme, such as the ability to draw evidence from a variety of sources to reach conclusions. These general skills are skills that are often looked for in most students studying other degree subjects.
Benchmarks of another twenty further subjects have been available for the last six months. They are widely different, and give very different impressions, but are some are surprisingly good. A lot of them are really well written. These are very interesting documents. If you want an idea of what your fellow academics' subjects are about or what they feel their subjects are about, then you'll get a remarkably good idea by reading one of these documents. When the benchmarking process was set in motion it was thought that the resulting benchmarks would lead to a type of national curriculum. . When you look at benchmark documents like those of Chemistry, then you can begin to see that they are indeed work into that kind of direction. However, on the whole when benchmark documents are examined, they do not lead into this direction, because people are quite anxious not to lead to a national curriculum. In addition a common theme emerges in that most documents are too vague in the first place.
The Biosciences Benchmarking Group was set up earlier this year. It was drawn up as a result of an originally rather ad hoc group. The members of the group were chosen following nominations. The criteria involved in the choice were factors such as age representation, gender and geography of the members, as well as the inclusion of a wide representation of biological fields. The members chosen and their affiliations are as follows:
The "Bioscience" Benchmarking Group"
Jeffrey BALE The University of Birmingham Paul BRAIN University of Wales, Swansea Darrell BROOKS University of Central Lancashire Sara CHURCHFIELD King's College, London Kathleen KANE University of Strathclyde Jacqueline LANDMAN The Nutrition Society Caroline MACDONALD University of Paisley David MALE Open University Roger MARCHANT University of Ulster Helen O'SULLIVAN Liverpool Hope University College Wendy PURCELL University of the West of England Jim RIMMER Aston University Robert SLATER University of Hertfordshire Janet SPRENT University of Dundee Simon VAN HEYNINGEN University of Edinburgh
Naturally, with the limitation of numbers of members, all fields of biology could not be covered by a representative member. For example, there was early criticism for the apparent lack of a genetics specialist. It was thought, however, that enough genetics and other subject knowledge should be present in the members chosen to cover these areas in the benchmarking process.
The QAA have set a two "key" questions for the benchmarking process. These include: · What are the ATTRIBUTES and CAPABILITIES a Graduate should be able to demonstrate? · What are the minimum standards of attainment in relation to those attributes and attainments?
Note again at this point that the interest lies in the minimum standards. Furthermore, the factual knowledge gained by a student is not necessarily the major point of interest. Attributes and capabilities are not the same thing as factual knowledge. Factual knowledge does come under the capabilities of a student, but there are clearly many other factors involved in this.
At this point in time, the likely headings for the Benchmark statement are: · Introduction · Defining Principles · Nature and Extent of Subject · Subject knowledge and Understanding · Subject Skills and Other Skills · Teaching, Learning and Assessment · Standards
The first meeting that has taken place so far was set up to look at these specific questions and headings. For this purpose the Benchmarking Group was divided into groups to produce extremely preliminary drafts for each of these headings.
In biology we have the advantage of a certain amount of previous information. This is because three years ago a preliminary investigation into this type of thing was undertaken by the Institute of Biology and the Biochemical Society to lead to a kind of Undergraduate Standards Programme. A detailed investigation was led into which skills, qualities and attributes various Departments of Biology across the UK felt they taught, were expected of a graduate in Biological Sciences and which of these were actually assessed. The core attributes of biological science graduates are published and available in the "Report of the Institute of Biology and the Biochemical Society to the Quality Assurance Agency of a pilot GSP Project" (November 1997).
To summarise, the expected attributes of a biology graduate are: · Critical reasoning · Subject's conceptual basis · Investigative skills · Communication · Data/information processing · Subject content and range · Laboratory skill / fieldcraft · Subject methodologies · Teamwork · Independence · Professional Skills · Time Management · Synthesis
The Attributes that are assessed are as follows: · Subject's content and range · Subject's conceptual basis · Critical reasoning · Intellectual analysis · Laboratory skills / fieldcraft · Data / information processing · Subject methodologies · Investigative skills · Originality
One important point to make here is that the expected and the assessed attributes are not the same. For example the first point in the expected list is critical reasoning, which appears 3rd on the assessed list. The subject's content and range appears first on the assessed list, whilst it only takes 6th place in the expected list. There is quite a wide difference between what is assessed and the qualities you would like or expect a graduate to attain.
No definitive conclusions on the content of each heading have been made yet. The preliminary drafts from the Benchmarking Group have only recently been handed in. It has been agreed, however, that there will actually be very little about the factual content required as a minimum standard for a graduate. This is due to the difficulty in deciding a reasonable minimum standard for a biologist. It is difficult to come up with anything useful and not extremely banal, e.g. that pants generally grow in the ground and a sheep has four legs, that you would expect every biology graduate to know, from the ecologist to the zoologists to the microbiologist etc. Instead the benchmarking statement for biology will follow the suit of some other subject areas and contain general skills that would be expected of a graduate. Examples of these are numeracy, literacy, experimental awareness, ethical surrounding etc.
Care will also have to be taken into what is seen as a prerequisite and background knowledge for a biology graduate. Does a graduate really require field and laboratory work experience? If so, then to what degree does he require these skills? How are they to be assessed? Would including a statement such as this as a benchmark exclude universities that did not have the resources or funding to support a lot of lab / field work. Are there not alternative means of teaching the skills needed in the laboratory? It will also be necessary to look closely at the levels of skills required for a threshold student. Do we really only want to include levels for a threshold student? Would it not be better to also include the level of skills required by a modal or typical student, which in other words would be a 2.2 or a 2.1 graduate. It is not really beneficial to only look at what is required for a 3rd Class Degree. However, we don't want to overdo this and swing so far as the Chemist have in compiling very detailed guidelines of what is expected for a 1st, 2.1, 2.2 and a 3rd.
During the benchmarking process the benchmarking group intends to include a wide as possible consultation with other groups and people. This would include not only discussion with formal consultation panels of the QAA, but also discussion with the people who applied to be members of the Benchmarking group but were but chosen, discussion with HUBS, other societies, industry (ABPI, Bioindustry Forum) and English Nature and similar. Many drafts will be written and amended. These drafts will be public and will also be available for viewing on a Web page. We will try to arrange a form of open comment through the internet on these drafts, but the logistics of replying to each comment needs to be solved first. Consultation will also be sought with other benchmarking groups in the fields of: Medicine, Veterinary Medicine, Dentistry, Agriculture, "Biomedical Sciences" and allied Professions to Medicine and Psychology. These are all benchmarking groups with which we overlap as the field of biology. There may be common themes amongst the groups, which would benefit from discussion and input.
I (the speaker) would like to finish of by asking if the word "Bioscience" is a real or even a useful word. The word "bioscience" is quite horrid, is not very useful and has been invented for this benchmarking process. I (the speaker) propose that the traditional terms "Biological Sciences" or even "Life Sciences" are better and more useful words for describing our field.
Is the benchmarking process worth doing? Will the Benchmark Statement be of any use to anyone? Will it only be useful to those teaching the subject? Is the Dearing Report the only real reason for going through this process? Is it just a bureaucratic exercise, bearing in mind that Benchmarking was initially intended for a variety of purposes, but now has been written mainly for other academics. The employers are no longer seem to be a target group.
I (the speaker) invite comments on these questions and welcome any views on what should be included in benchmarking.
Discussion:
Paul Brain: I would like to bring up the issue about defining benchmarks for the Pass degree. We all compensate in relation to performances of students to a certain extent, even though the QAA does not like us to do so. How are you going to specify benchmarks when you are still going to allow a student to pass because of very positive performances in one area compensates for poor performances in another area. Reply: The official answer is that you shouldn't really let any student graduate who has not reached these minimum levels. However, this goes back to the question I asked earlier as to whether these minimum levels are actually of any use.
Kevin (Abertay University, Dundee): I am interested in the idea of identifying modality and the idea of a 2.2 -2.1 boundary. Institutions could end up somewhere down the line that these documents may be given to much importance and may lead to external examiners not allowing institutions to award a particular student a 2.1 because of these standards. Are there going to be any safeguards against this?
Reply: I agree that this is something one could worry about if these documents were given too much importance by external examiners. I don't find the assurances given by the QAA particularly confident on this issue. I mean, it is possible that the QAA might cease to exist in the current way and we would be left with the same problems under a different "regime". I don't think, and these are my personal thoughts, that we will be using the terms 2.1 -2.2 boundary. There is the idea that to look at the benchmarks only in the scenario of the "just Pass" student does not really provide a particularly good model.
Jim: This is comment rather than a question about one of the positive things about the benchmarks. I have read the Classics benchmark and even though I know nothing about classical education, I found it very interesting. It is in fact something that might have inspired me to study Classics if I had bee a student who did not quite yet know what I wanted to do. I think if it is done well, the Benchmarking process in the Biological Sciences will be a positive thing. If done in a positive way it can highlight the importance of the field of Biosciences. I think this is especially important because the notion/impression still persists that we are a second class science. It is not real science such as Chemistry and Physics. If nothing else the Benchmarking process will also hopefully present a more coherent view of what the Biological Sciences are about. We are not megalomaniacs intent on creating Frankenstein foods, or even worse, a bunch of incompetents playing as God and meddling with things that we shouldn't.
Paul Brain: Following on from what Jim was saying, there are other potential benefits from the exercise. The process allows us to reflect on issues more widely and direct students to think about the ethical and moral issues of studying biology and that this itself is quite an interesting issue. Also, given the contrasting views on resource, some positive statement about the minimal involvement of practical work may be useful.
Simon van Heyningen: Can I just ask a question about that, not that I don't agree with you. One of the things that benchmarking panels tend to try to do, is to produce a benchmark that everyone around the table is going to be able to meet. In a way this is incompatible with what you just said. Is it an argument " No, you can't put that in, because universities x and y won't be able to do it". How does the community feel that we ought to go about this? For example, I know I wouldn't want to say what kind of practical work should be prescribed, but I agree that there should be some.
Speaker Trevor ?: I think the benchmarking exercise is good in that it makes us think about exactly what we teach and how we go about it. The worrying thing is that it doesn't seem to be set in a very appropriate context, with things happening around us like Curriculum 2000, Foundation Degrees and profiling degree classification. There is a danger that benchmarks will be used for other things that they were not intended for. This is when people will start saying that it was not a particularly good exercise.
Simon van Heyningen: I agree with that very strongly and it goes back to my point that all these things are being done far too quickly. Clearly, the benchmark forms a sort of level descriptor for the Honours level. Also, reading my colleagues' drafts on the assessment section, there are many overlaps with the Code of Practice on Assessment. All this needs to be done together and everything we produce now will be a first draft.
Trevor?: I think the sector really needs to concentrate on the issue of degree classification versus profiling. This should be incorporated into benchmarking. For example, if we look at what will be particularly meaningful for employers, then surely profiling is the way forward. We have been talking about it for years, but we don't seem to be doing anything about it.
Simon van Heyningen: My view is that I am particularly in favour of profiling. About 80% of information about a student is thrown away when we issue a degree class. There seems to be great agreement across the sector that profiling is something we really need to address.
Sally Brown: Having read many benchmarks I would like to highlight that there is a great deal of commonality between different subject's benchmarks in that they very often focus on generic skills. There is even the notion that it is not possible to write a benchmark unless it is in this generic fashion. Thinking about the generic skills that a particular degree promotes is actually the most useful part about the benchmarks. It should not be used to develop a national curriculum, I am not interested in that. However, I am interested in the description of things that make a graduate different from other non-graduates.
Clive Kennedy: There is a point that never seems to be incorporated into benchmarking processes and that is the public perception of what is expected of a biologist. By this I do not mean the newspaper press perception, but that of your neighbour. What they would expect you to know.
Simon van Heyningen: No, we would not put something like that into a benchmark. For example, my neighbour expects me to be able to identify each flower species, and that is not what being a biologist is about. We need to remember that we are defining benchmarks currently for academic purposes. The public is a stakeholder, but we cannot currently simultaneously write a suitable benchmark for the academic and public sector. In the future, there may be one for the public.
Jim?: One question from the public that may be useful to answer is: "Why do we need biologists -what do they do?" The answer to this question ought to appear in the core of the benchmarking process. We have reached the stage of human regulation when we can alter plants to perform better in the world we have altered. We need microbiologists because penicillin does not work anymore. Addressing these issues and the context of biology in society should be at the core of the benchmarks, so that the benchmarking process is not just another dry and sterile academic exercise. We need to make the public aware of the practical benefits.
Simon van Heyningen: I agree that the public are stakeholders that we should be aware of and that they should be made aware about the role of a biologist, but they should not be allowed to define it. If they do not know and understand what we do, the result is that they will be scared. We can see this so clearly in the GM scare.
?: Maybe we could include a paragraph in the benchmark document addresses to all stakeholders explaining the importance of biologists and include list of things that we do.
?: We can gain some insight into the public perception of biology by reading UCAS forms and looking at the aspirations of students applying to study biology. It is especially interesting to take note that their aspirations are not necessarily covered in the courses that are offered at universities.
?: Somewhere in QAA there surely must be hundreds of sets of programme outlines and objectives from different universities. Can these not be somehow collated to provide a skeleton for the benchmarks? Can we not pull out the benchmark points from the QAA review?
?: The overview reports could be used for this.
Professor Sally Brown "What's ILT All About? What's in it for me and my staff and what processes are involved in joining?"
The Institute for Learning and Teaching (ILT) was launched in 1999 as a result of Recommendation 14 in the Dearing Report. The first members of staff were appointed and the offices were opened in 1999. The first members of the ILT were admitted in October 1999. By July 2000 the Membership Services of the ILT were in full swing. Detailed information on the contents of this talk can be found on the ILT leaflets and also on the ILT website.
The ILT was set up for three main purposes: 1. To enhance the status of teaching 2. To improve the experience of learning 3. To support innovation in higher education.
1. Dearing recognised that the status of teaching needed to be enhanced. He recognised that different universities had differing slants on approaches taken to teaching and learning in relation to research. These approaches ranged across the board from mainly teaching orientated universities to those that focussed mainly on research, with teaching being of secondary importance. Teaching in the latter type of universities was only as good and effective as time allowed, which was merely what could be spared as a minimum of time away from research, which was perceived as far more important. 2. Point 2, to improve the experience of learning is a point that cannot be undertaken by government dictate. The only way the learning experience can be enhanced is through the individual commitment of lecturers. This is one of the main reasons why the subscription of membership is a matter of choice and cannot be enforced. 3. The ILT helps individual lecturers keep up to date on innovations in higher education.
5 trends emerged from the responses to the initial consultation report whilst the ILT was being set up:
1. ¾ of the sector wanted an ILT to exist. 2. 2/3 of the sector wanted membership to the ILT to be a demonstration of threshold experience and not just a matter of paying up a membership fee. 3. It was initially thought that membership to the ILT should be by portfolio under 24 headings.
4. Categories of membership were discussed and changed. 5. The initial requirement of the ILT programme included a section on "Continuous professional development". This involved attendance on 4-5 workshop days. This was finally deemed as too demanding and was revised.
There are currently two routes of entry to membership of the ILT. Details are outline in the ILT leaflet and on the ILT website, and these will be summarised here. Until September 2001, academics with three or more years' teaching experience (pro-rata for part-time staff) can apply directly by providing a summary of professional experience in five broad areas. The submission has a 1500-3000 word limit. Two references are required to support the submission. The five areas of expertise to be highlighted are: 1. Teaching and /or supporting learning 2. Designing and planning learning activities 3. Assessment and provision of feedback to students 4. Developing effective learning environments and student support systems 5. Reflective practice and personal development
This route currently represents the most significant route of membership. Other routes exist, such as a one-year qualification of a Postgraduate Higher Education Teaching and Learning Certification.
There are many incentives for becoming a member of the ILT. Firstly, membership of the ILT represents a type of kitemark of experience and expertise in teaching and learning support. The ILT provides membership of a community of learning through which you can share practice experience and new ideas with other professionals in the field. Membership to this new organization gives you a chance to shape, influence and direct the service of the ILT. Membership of the ILT also provides a performance indicator for external scrutiny. For example, the QAA is currently waiting for the ILT to reach a certain threshold of membership numbers and status, after which point membership of the ILT will be used a performance criterion.
For this reason some universities are currently offering their staff inducements to join, such as rewards in terms of pay and promotion; links between ILT membership and successful completion of probation for new members of staff; payment of the processing fee and the first few years of membership. The current cost of membership is an initial processing fee of £25 (70% of institutions pay this) and an annual fee of £75 (35% of institutions pay this).
There are many benefits of membership, which include: · The "Package": an international journal (Active Learning in Higher Education) twice a year; a newsletter three times a year; access to a Website with a members' only area and networking opportunities (from 2001) · Discounts at ILT events and on a range of goods and services · Gain recognition for their expertise in supporting student learning · Membership of a professional body with the kudos of accredited status · Keep the formulation of standards of practice where they belong - in the hands of the practitioners Discussion:
?: I have now written references for two colleagues. They seem to have included a large amount of supplementary material in the form of practical courses, course questionnaires completed by students etc. Would that be absolutely necessary?
Sally Brown: No, all that would be sent back. Additional materials will not be looked at. The things I outlined in this talk are the only things that are required. The process is very straightforward. Each application is looked at by two external acreditors, who are not staff of the ILT. They will only look at the material you are required to prepare. If they are satisfied that what you prepared meets the criterion of the ILT, you're in. If not, approximately 8-12% get referred. Your material then goes to two further examiners and you get feedback of what you would need to do if you were to reapply. The most frequent reason why people get referred is because they tell us what they do, and not how or why they do it. The document needs to be evaluative. There is a lot of information out there on how to go about this, and I would particularly recommend to you the Southampton website on how to join the ILT.
?: What is the processing time for applications?
Sally Brown: The maximum turnover time is 12 weeks, except when there are referrals or when incomplete applications are handed in, which obviously delay the process. 8 weeks is typical. The whole process is generally delayed by the fact that we have panel meetings fortnightly - the accreditors literally come in once a fortnight and spend a day looking at the applications.
Chris? University of North London: Could I just add on the behalf of my school that we were very impressed by the process. The processing time was generally short, the ILT was very good at providing help about uncertainties in filling out the form and feedback..
Sally Brown: The University of North London is one of our top 5 runners. Of the top 50 institutions half and half are old and new universities, which was quite surprising to me. We have been told that we are quite good at dealing with enquiries. You can just call up, and if you don't get anywhere the first time, ask to speak to me and I'll make sure that you get an answer. We are running out of time for questions, but if you do have any questions that have remained unanswered then please do e-mail me. My e-mail address is: sally.brown@ilt.ac.uk. You have also got the website that you can look at.
Professor David Ross (University of Abertay, Dundee) "Applying ILT Policy and Practice: an Accredited Teaching in HE course"
There are many reasons that led to the desire to develop our teaching skills at Abertay: · integral to University Mission · "good" practice in staff development · changing environment of HE · enhancing student performance · TQA evidence · emerging QAA Framework and PI's · moves towards Accredited Teaching and Learning qualifications and courses for staff
Qualifications in Teaching and Learning provide recognition of experience and ability as a teacher, as well as self-development. The qualifications offer promotional prospects and other rewards. It is anticipated that within 5 years' time these qualifications will become "currency", possibly even requirement. Qualifications in teaching are an internal statement of quality and as mentioned before they will increasingly be used as a QAA performance indicator.
The planning goals of the Postgraduate Certificate in Higher Education Teaching (PGCHET) were: · lead to 60 SCOTCAT Level M points · flexible timescale over 1- 4 years (typical duration = 2 yrs) · no cost to individual supported by School · limited face -to- face contact sessions · reflective practitioner theme · strong emphasis on practice and evidence generations by Portfolio · APL/APEL route available for experienced staff (60 % of course can be completed through APEL) · mapped onto SEDA/ILT criteria
The PGCHET has five main modules, all of which are assessed. · TLA Theory, Practice and Context (Basics) · Curriculum Design and Teaching Delivery · Assessment Methodology and Practice · Computers and Information Technology · Enhancing Student Learning
The core content of the PGCHET is delivered by a series of focused Seminars/Workshops. This is reinforced by Guideline Papers and recommended reading. Attendance at and participation in seminars is key to successful development of reflection/evaluation skills. The format of the PGCHET allows for "Individualised Learning Contracts". Every participant following this route negotiates a Learning Contract with the course director. The Learning Contract essentially an agreed programme of assessment. It can include learning outcomes achieved by "Action Research". The whole programme is supported by mentors within CELT and cognate subject areas.
The PGCHET at Abertay is ILT accredited. This accreditation process involved 2 main stages. Stage 1 consisted of the internal development and validation of the PGCHET at Abertay (June 1998 - August 1999). During this time it was debated whether accreditation through the older SEDA or the newly established ILT should be sought. It was finally decided that accreditation of the course would be sought through ILT. The next process then involved mapping of the UAD PGCHET learning outcomes to ILT professional activity areas. This process took longer than anticipated since the ILT areas were also still in development. Every time these were changed, the outcomes of the PGCHET needed to be altered as well. Stage 2 then comprised the final ILT Accreditation of the course in March 2000. The ILT Professional Activity Areas mentioned above are: · teaching and/or supporting learning in higher education · contribution to the design and planning of learning activities and/or programmes of study · provision of feedback and assessment of students' learning · contribution to the development of effective learning environments and student support systems · reflection on personal practice in teaching and learning and work to improve the teaching process
The requirement for ILT accreditation is the submission of a basic course document with a mapping commentary and a limited amount of supplementary information. Once submitted, a team of 3 accreditors is appointed (after consultation) by ILT. This team identifies issues (5 in the case of Abertay) that need discussion and possible amendment for accreditation 1 week in advance of the visit. This allows you to prepare your response. The visit is then conducted in a non-confrontational spirit of mutual benefit. The outcome in Abertay was successful with one condition, which was to provide the ILT with teaching portfolios within three years.
There are currently several routes to membership of the ILT at Abertay. The first is the "fast-track route" of direct application to the ILT for members with 3 or more years' of teaching experience, as was outlined in the previous talk by Sally Brown. The second route it through attendance to and completion of the PGCHET course offered at Abertay. A modification of this second route is the AP(E)L route and entry/gaining of the 5 modules in this way. This is designed for more experienced teachers to be able to gain entry into the course somewhere along the course of the 5 modules, having completed up 60 % of the course through a portfolio. Through the two latter routes staff gain a postgraduate certificate. At the end of this there is a kind of "equilibrium process", where those with the PGCHET gain membership to the ILT and those with membership to the ILT can top of their membership up with the PGCHET.
Was the whole process of ILT accreditation of the PGCHET course at Abertay worth it? The PGCHET is a valuable qualification in its own right and goes beyond ILT requirements. Mapping the course to ILT was difficult at the time due to changing ILT criteria. The accreditation process itself was smooth, positively constructive and easier than internal validation! The real proof will come when the first cohort of students completing the PGCHET course qualifies in June 2001.
The current status of the PGCHET is that more than 50 staff members are currently enrolled over two cohorts (these numbers are ahead of expectation). The drop-out rate is low. There is, however, already some conflict regarding staff-time to participate. Staff often cannot find the time away from teaching to attend the courses, so progress is slow. This years' cohort contains some new staff still on probationary contracts. The course offers the benefit of focussing the mind on teaching. Early assessments have been found (on average) to lack evaluation and reflection and tend to be more descriptive.
The probationary contracts of staff have needed to be worded quite carefully, since membership of the ILT cannot be demanded. The usual wording is that "staff should undertake the course in view of joining the ILT in future", or similar.
Discussion:
Paul Brain: Who delivers the teaching of the 5 modules?
David Ross: I am course director, as well as, the director of CELT. I deliver the main bulk of the seminar delivery. I have about six people in staff with different areas of expertise that I call upon to teach certain aspects of the course. I do on occasion call upon external speakers from other universities, when I feel that what they have to offer is of benefit to our staff.
Wendy ?: What is the impact on student learning? How do you plan to say whether your course has resulted in the enhancement of student learning?
David Ross: We are now asking our Academic Schools to reflect much more than the internal QA system prescribes on the whole learning experience of the students and to map this on to how many of their staff are taking or have completed the PGCHET, have membership to the ILT and for them to come up with evidence of examples of where they think student learning has been enhanced. It is too early to say yet, though. I do know of one example of a lecturer who has said that the learning of his students has been enhances, simply because he takes a more structured approach to teaching and so on.
?: I would like to fill out a little more about these things being done on a multi-institution basis. You said, and I understand this, that a lot of what is taught is institution specific. On the other hand, colleagues, especially those who are reluctant to go into this kind of thing, think that it may be better if it were taught on a subject specific basis. There are things to be shared, even between institutions of different sorts. I was wondering what you thought about this.
David Ross: We are already talking quite seriously with a sister university in Dundee about delivering the core content of this course at the same time. We went for the ILT accreditation about 1 week apart from each other. In Scotland, there already is quite a large amount of co-operation in educational development. It would be logical to see a kind of core content identification.
?: I personally think that there is far too much reinventing the wheel going on I this kind of thing, with different institutions trying to achieve the same thing.
Sally Brown: I would like to add two things to that. My experience on teaching and speaking to people on these types of courses is that there are two types of responses. Some people say that the most beneficial thing to them was the contact with people from different subject areas. The other half say they would have benefited so much more if the course had focussed only on their subject and not included people from different areas. Of course, these are not mutually exclusive and I believe this why the funding bodies have gone in the way they have gone about the support they have given these types of courses. I would like to add that there are 24 subject centres who's specific remit is to disseminate and develop good practice and materials on learning and teaching. In what I hope has begun to show joined up thinking the management of these subject centres has been handed over to the ILT. We do not own them, we simply manage them. So there is a strong subject specific development and those subject areas will we aware and active in the community for the development in teaching in learning in your specific subject areas. They are supported by the Generic and Technical Centres, which are also based in York.
?: When I talk about this subject with friends and other lecturers and especially with colleagues above the age of fifty, there is often the notion that you cannot teach old dogs new tricks. What is in all these developments for the older lecturer who perhaps will be retiring in 10 or so years?
David Ross: I can only give you a personal view from the Abertay side in that ILT membership and relevant courses and accreditations are more of an official recognition of all those years of experience and dedication in teaching. It is not only about teaching an old dog new tricks, but at Abertay we believe that you can teach an old dog new tricks if this is beneficial.
?: I would just like to add that at our institution, the more experienced lecturers have contributed greatly to others through their experiences in the field of teaching.
Sally Brown: I was once asked by a senior lecturer: "What is in the ILT for me?" and I replied by asking what he could offer the ILT. Also would it not be a shame that after spending and dedicating a life-time to teaching you did not receive an official recognition of your efforts. It is not just a case of teaching old dog new tricks, its is also a case of giving credit where credit is due.
?: What is the age distribution of the current members of the ILT?
Sally Brown: I think it is too early to say that, as we do not have the breakdown yet. Also we run into the difficulty that we do not as for the age in the application, because of our equal opportunities policy. We can tell al lot from the job descriptions, though. I would say that there is quite a good mix. We are also getting more members through direct application than through accredited courses.
?: I would like to ask how you decide who is exempt from attending the actual PGCHET course. You say that it depends on experience whether all 5 modules must be attended and that those with some experience can complete some of the course by preparing a portfolio. New staff must attend the course. How do you decide how much experience allows for exemption?
David Ross: I say to people in that situation that if they think they are experienced enough to join the ILT through the fast track route of direct application, then they should do so. This route is available until September 2001. What I do is go into individual consultation to decide what is the best route for the individual of concern. |
|
Charity # |
|
