Posted by John Kleeman, Founder and Executive Director
The Association of Test Publishers (ATP) held its annual Innovations in Testing conference (virtually) last week.
There were 168 sessions including an excellent keynote on disruption from Jim Harris, 1200 attendees, and a lot to take in.
Here are some of the things I learned and want to share.
1. Virtual conferences can work.
We are all over-Zoomed, but the ATP conference was exhilarating and enthralling. Key factors why this was so:
- The standard of sessions was very high
- Sessions were short and mostly live
- There was a central hub “activity stream” which allowed discussion
- And there was a lot of interaction, discussion, talking during sessions
- The attendees were interested, engaged and enthusiastic
There was a huge amount of work by the professional team organizing (thank you Designing Events) and by the volunteers (thank you Ada, Isabelle, Brodie and many more), and that was also critical.
2. Online proctoring is here to stay.
There was even a debate on whether brick-and-mortar test centers will be relevant in the future. I’m sure they will, but everyone knows that giving test takers the convenience of taking tests with online proctoring is a big part of the future. Yes, some tests will still take place in test centers but the question is not whether online proctoring will be used but which kind of online proctoring (human, automated, AI with human in the loop and so on). And also how to do online proctoring conveniently, reliably and consistently and for more types of exams.
3. Assessment people do care about diversity.
The theme of the conference was “assessment for a better world” and there seem lots of people in our industry who care deeply on making tests fair and free from bias. There were several sessions on diversity, one I attended was run by John Weiner in a “peas in the pod” format (see picture right, thanks cApStAn for capturing) trying to unpick what we mean about fairness and diversity.
Part of the truth seems to be that “test scores don’t create inequities, they reflect them”, but most people I spoke to at the conference seem very committed to diversity initiatives to improve the equity of testing. For example, there were sharing of book lists to help understand diverse viewpoints.
4. AI in assessment is moving from marketing to reality.
Several testing companies use artificial intelligence (AI) in production already and many more are looking at it. I hosted a panel session with two testing experts (from Duolingo and Caveon) and two external experts (from Riiid and Microsoft) (see picture, thank you Tim Burnett for capturing). There were also other sessions on AI. There is huge promise in AI – including in making test scores more accurate, making learning better and increasing fairness and removing human bias. But AI is not fault-free and also, it’s difficult to explain and be transparent about how AI makes decisions. There is a large risk of AI bias and mistakes, especially as we learn. The future looks exciting and the ATP will be at the forefront of defining best practice and seeking to find the promise and avoid the peril.
As a side comment, one good thing to see was that there was huge attendance at most sessions, more than there would likely have been in person. For example over 140 people attended this panel session, even though there were several other sessions running in parallel.
5. Test anxiety is real, but can be ameliorated.
The session I enjoyed most was one by Ben Lovett of Columbia University who talked about test anxiety. Some things I learned:
- Everyone experiences anxiety to some extent, but some people do get very severe test related anxiety.
- Psychologists can measure anxiety by various rating systems or structured interviews and help identify people who have such anxiety.
- A common accomodation for test anxiety is to give more time to take account of time needed for coping strategies or anxiety.
- Sometimes such test-takers don’t use the extra time, but merely having it reduces their anxiety.
- Other accomodations include a separate room or giving test-takers more control over the testing UI, e.g. the ability to change text font size can help.
If we are testing for a better world, it seems important to do so in a way that minimizes anxiety.
Of course, there were lots and lots of other areas covered. All the sessions were recorded and will be made available shortly so I look forward to catching up with some of these. I believe they’ll also be available for others to watch as well.
I hope this summary was useful. Of course, we missed meeting in person, but we did the best we could in virtual and still found it engaging and had great learning.
As well as my role at Questionmark, I’m the ATP 2021 Chair. The ATP is the non-profit global community for testing and assessment, and I’d encourage readers to consider submitting a paper for our next conference (European ATP, in September), to join us as a member and for sure to come to our next Innovations conference which we expect to have in person in Orlando in March 2022. For more on ATP, see www.testpublishers.org.
For more on Questionmark, please request a demo today.
John is the Founder of Questionmark. He wrote the first version of the Questionmark assessment software system and then founded Questionmark in 1988 to market, develop and support it. John has been heavily involved in assessment software development for over 30 years and has also participated in several standards initiatives.
The post Five things I learned from the ATP conference last week appeared first on Questionmark.
