Examview Results and Reports

So... there's a lot of stuff to wade through today.






1. Big disappointment - Sharepoint was out the last couple days. So I routed my kids to review activities through Quia. But that means I can't track any meaningful Sharepoint usage data. I may keep this week anyway. But since I emailed the kids to go around Sharepoint, I won't really know whether many took advantage of this option. If I add this week, the Sharepoint numbers will look low, so if scores stay stable or even go up, it'll look like Sharepoint is inconsequential. On the other hand, if scores go way down, it will look like the lack of Sharepoint really hurt their grades when many students may well have used the online activities and still suffered grade reductions. There is a hit counter on the quia site which is at 237, but I didn't check it prior to yesterday. Well, I'm glad I had quia! I at least had some other way to get the kids their resources. But a lot of parent emails are outdated, too.



2. Speaking of Quia, I'm excited about the upgrade next week. I'll keep you up to speed on that.



3. Examview Reports. I've always liked the detail that you can get from examview reports if you code your questions well. I used to think coding the questions was a lot of tediuos work, but if you have a lot of questions with the same criteria then you can "Duplicate question" and then just change the stem. EVEN BETTER, though. I use MacroPro to go edit and code bunches of questions. So I can still use old questions that I didn't code well or other people's questions and add my codes to them. Then I get more useful reports. For example, I give a quiz weekly on the vocab, grammar, and notes that we've covered in class. In order to find out how a kid does on vocab or grammar or notes, I used to have to give the quizzes separately. Now I just call those major headings National Standards and can get a simple report like this:

Well, you can't really see that, can you? But it gives the class average on Notes, Grammar, and Vocab as well as the number of students performing above a threshold (in this case, 70%) in each area. You can also print out the names of those "in range" students who are not at or above the threshold.





But that's not all... Within any one of those "national standards" there are state and local standards. For example, in Grammar I have the state standard "parts of speech" and the local standard "nouns". But I can go even more specific than that! Using the Learning Objectives code, I can add a tag like "Common/Proper" "Singular/Plural" etc. to the nouns standard. When I run reports at any of these levels, then, I can find out who is doing how well on each specific skill. I'll print out another report you can't see. I should figure out how to do this better!




Through the haze you might be able to make out our struggles with pronouns (so many kinds!) and our peace with nouns. Once again, you can print the names of students who need more work and then you know exactly where to focus the students' energies.

But all is not perfect in Examview land. When you go to take the next step - Initiating interventions to improve performance - you hit a snag: How do you use examview to track one standard across multiple assessments? After all, if I can't easily figure out the difference in someone's scores from one day to the next, I can't measure the value of the intervining interventions (ha). I could probably export all this to excel, but I'm not working on a stats degree here! Don't get me wrong, Examview will provide a report of a student's TOTAL performance on a learning objective or standard over time (you can enter a date range and get an average across that range). But you can't get waypoint measurements to track change over time. PLEASE, PLEASE Examview listen to me on this one. Teachers want to know that our interventions work, and we need a tool that lets us measure these fairly easily. Examview has the numbers we need for this, but not the subroutines to crunch these numbers in a way that we can use.

Well, that's enough for now. Enjoy the weekend...

Wow - Time flies

So, I meant to continue that last posting but ran out of time and then had two days out with sick kids and now I'm running around catching up. But here's the a screen capture with pretest, sharepoint usage weekly, and weekly quiz scores. I've cut out names and id numbers for privacy.

Oh, exciting news, too. Quia is making activities trackable starting next week, so students who do more quia than sharepoint review guides can figure into the mix. I may even be able to compare the two...

Gott run

Sharepoint, weekly scores, coding in examview

I'm on the wrong computer to be blogging, but that's gonna have to work. On the other computer, I have MacroPro updating the coding on a bunch of Examview questions.

Part 1 overview - I hope, within the scope of today's blog, to make it clear how the coding can help me figure out what kids know and don't know. And I'll also critique how the codes let me down in a few cases. And I may vent about a shortcoming of Examview that I'm begging them to fix. So, some cool stuff, some mistakes on my part, some wishful thinking about how a product could be improved.

Part 2 overview - I'll post some info about how I can compile Sharepoint/Examview data to measure weekly visitation and progress on skills-based tests. The blog is already helping me here, in that I have thought about how to get weekly data so that I can see how the kid does from week to week and in comparison to the pretest. Now my problem is that I don't know how to crunch the numbers. So, a step forward and a problem.

Part 1 - As Examview is superlative for objective assessments, my most frequent applications are for quizzes on vocabulary, grammar skills, and class notes. I like to give a quiz each week that measures performance on new skills and knowledge and retention of what we have learned in the past. Since Examview banks let me store lots of questions, I can generate tests easily and create tests from a combined set of banks. Each Friday I give the students one of these combined tests and use the results to figure out what we've learned, what we've still got to work on, and how quickly I should move forward or tread a little water. Obviously, if I just got the overall results on such a unified quiz then I wouldn't know much more than overall success (they're doing pretty well vs. they are doing great on vocabulary, good on notes, and poorly on grammar - with a particularly disappointing grade in pronoun usage). So, for each examview question I add information in these little boxes for learning objective, national, state, and local standards because those are the data tags that I can run reports from. So I'm going to switch computers and continue this on the next entry b/c I can add screen shots to clarify and give examples.

After a long absence...Welcome Back

So I ran out of time, energy, motivation after about four months of blogging last year, but now I'm back and hope to keep a good account of the technology interventions employed throughout the year.  As an introduction (starting completely over) the hardware that I have available for my students includes 27 wireless tablet PC's (new this year), an overhead projector connected to vcr/dvd/stereo receiver, a class set of eInstruction's CPS remote response pads, headphones, a digital voice recorder... I think that's it.  That's pretty sweet!  

In terms of software/webapps, I have all the Microsoft standards and use Microsoft Sharepoint (a District standard) for sharing information and course content with students and parents.  To the limited degree that I use it, the Sharepoint site Discussion board serves as our social networking.  I use Examview 6.0 for in-class testing and to create activities for the remote pads (which are delivered via CPS and Quizshow).  Online, Academic Reading students use PLATO for targeted instruction.  There are also numerous review games posted on Quia.com for kids to use.  The whole lab is snugly blanketed in GenevaLogic's Vision software for lab monitoring - an awesome tool for keeping students on task and for speeding up the delivery of content on the computer.  The district also employs Deepfreeze for the protection of our computers.

Oh, and our textbook, HRW's Elements of Literature, is online at my.hrw.com.  Our vocab book, (Sadlier-Oxford's Vocabulary Workshop Level D) has limited activities online. We have no grammar book per-se at the freshman level, but I try to follow along with the early chapters of what is now HRW's Holt Handbook and used to be Warriner's.  And for sure we have novels of all kinds.  We've also got a ton of audiobooks both within English and in the Library.  Well, not a ton, but a growing collection.

As for human resources, I try not to name names or anything in here.  My partner in teaching an and learning works in the same class environment as me and we often work together and sometimes go separate directions when developing materials and activities.  Like a character in a Russian novel, I'll call him D-.  In tech support, I'm lucky to have the ongoing assistance of someone who has been willing to come early, stay late, work weekends, and image and re-image machines while we troubleshoot problems.  I'll use the name Kelty because we've talked camping a bit (I'm an avid camper and could digress for hours on where the family has been and would like to go).  I have a tech release period along with another tech-type guy (though he knows a lot more than I do) and he'll probably come up so I'll refer to him as Oak.  The only two people ever likely to read this (a department chair and assistant principal) are both good resources but I won't mention them often because it would be weird to evaluate someone who is writing about you.  (unless they score cool resources for me, in which case I'll give them mad props - Thanks for the Listening Library CD's WoW.).  

At any rate, if I wanted to state my purpose for exploring technology as a means of improving student learning it might take a little time.  Essentially, I'm looking to increase engagement, target learning, and improve reporting.  That said, the process should be recursive so that better reporting enhances my ability to target assignments, and assignments that give a student the right challenge at the right time are more engaging than those that are too easy or unaccessible. To this end, I have had to learn a little more (last year) about numbers.  At first, it was a struggle figuring out how to get numbers that provide more and more specific data about how well students are doing in reaching specific learning goals.  Once I finally got  numbers to do that, I had to try to figure out what (if any) meaning could reliably be derived from such statistics.  After a year of this self-guided study, I know a bit more about what kids know and can do prior to a lesson and afterwards.  I also can monitor some of the interventions that they have taken in order to prepare for assessments and can judge the efficacy of some of these interventions. 

If those last couple sentences sound a bit loose, you're spot on.  The whole point of the blog (and the work that I do behind it) is to examine what I do know about student learning, question the degree to which I really know it, and deliberate whether there's any substantial benefit to students from such knowledge.  Sometimes I learn silly things, as with last year's exposee on the difference between my perceptions of how well a class did vs. their actual performance.  I kept rating my period 10 lessons as less effective, but the remote control data collected showed the students (an energetic bunch at the end of the day) were learning just as much as my more docile students in period 2.  They looked and acted less scholarly, to be sure.  But they learned just as well.  

Perhaps an example would help - I can take the site usage statistics from Sharepoint and see who has visited the site more often and compare that against the performance in the class to seek a correlation.  Last year, I did that at semester's end and found a positive correlation.  BUT, what do I really know?  Since  kids who are more likely to go to a class website are probably the kids who are more likely to do homework, study, etc, it seems almost inevitable there would be a correlation (meaning it's possible that all my sharepoint stuff is a waste of time... or not).  But this year, I've pretested all the kids.  Soooo, I should be able to measure improvement vs. Sharepoint participation which seems like it could tell me a more valid story about the effectiveness of using a Sharepoint site bolstered with numerous supplemental learning activities.  Furthermore, week to week site usage statistics can look for correlations between the ups and downs of those students who perform with dramatic inconsistencies.  Are trips to the Sharepoint in line with those ups and downs?  I can look.  

The details of doing such an experiment are enjoyable to about a half dozen nerds in the world.  But in the end, I pull data from Sharepoint into excel and drag test results from Examview into excel and the rest is history.  In fact, I've got a ton of review stuff up now and I know some will ignore it and some will use it, so tomorrow I'll run a correlation and post it back here.  For now, I've got stuff to do....