Behavioral Experiment Software Survey Results
Hi everyone,
The results of the software survey are in. We had 187 responses, but one was unanalyzable (the respondent did not specify a software package). Thanks everyone for your responses; I hope these results prove useful.
Here's a one-paragraph summary of the survey results; details below: E-Prime is the most popular package of those surveyed, but the majority of folks are using either E-Prime, DMDX, or some flavor of PsyScope. E-Prime, PsyScope, SuperLab, and ERTS are all rated as easy to build experiments with, and about equally so. DMDX and NESU are seen as slightly harder. Presentation and MatLab are notably the hardest of the commonly used packages. SuperLab was seen as easiest for novices. E-Prime and PsyScope were rated a shade harder for novices, and then from ERTS to SuperLab Pro to DMDX to NESU, novice-ease ratings dropped. Presentation and MatLab were both seen as notably difficult for novices. DMDX nets the highest satisfaction rating, with PsyScope X and E-Prime a smidge lower. MatLab, SuperLab, and Presentation rank below that, ending with NESU, PsyScope classic, and SuperLab Pro. PsyScope X has the highest "sticking with" rate, and E-Prime, SuperLab, MatLab, and DMDX all have 50%+ "sticking with" ratings. People are running away from PsyScope classic in droves, probably because the classic Mac OS is very quickly approaching end-of-life. E-Prime, both flavors of PsyScope, and DMDX are highly recommended. MatLab is also well recommended. Then it drops noticeably to Presentation, SuperLab, ERTS, and finally SuperLab Pro and NESU. A summary of two open-ended questions is also included below.
*
Details:
Of the 186 valid responses, the most commonly used software package ended up being E-Prime (57 responses, 30.6%). Here's the ranking of software packages, listing number of respondents and percentage for each:
1. E-Prime: 57, 30.6%
2. DMDX: 32, 17.2%
3. PsyScope Classic: 18, 9.7%
4. Presentation: 12, 6.5%
5. PsyScope X: 11, 5.9%
6. NESU: 8, 4.3%
7. ERTS: 6, 3.2%
8. SuperLab: 5, 2.7%
9. MatLab: 5, 2.7%
10. SuperLab Pro: 4, 2.2%
11. Linger: 4, 2.2%
12. MEL: 3, 1.6%
13. Experiment Builder: 3, 1.6%
14. EyeTrack: 2, 1.1%
The following packages had 1 response each: Authorware, C programming, Delphi Borland, DirectRT, ExBuilder, Habit, Inquisit, MacroMedia Director, PHPsurveyor, PCexpt, RSVP, WWStim, WebExp, iMovie, tscope, vision analyzer/recorder.
Bob Slevc worked to dig up links for many of these software packages. I'll put the links below the signature line.
Note that PsyScope classic and PsyScope X could be combined to have a total response count of 29 or 15.6%, which would keep it in third place behind E-Prime and DMDX. SuperLab and SuperLab Pro (which I assume are distinct) could similarly be combined for 9 responses (4.8%), which would put it right behind Presentation.
*
We asked, "How easy/hard is it for you to build an experiment with your software?" with a 7-point response scale (1 = "very easy" and 7 = "very hard"). Overall, the mean difficulty rating was 3.09 with a standard deviation of 1.43, and a median of 3. Here are the mean and median build-difficulty ratings for the packages that received at least five responses (combining the above noted PsyScope and SuperLab):
ERTS (6): 2.5, 2
E-Prime (57): 2.68, 2
PsyScope (29): 2.86, 3
SuperLab (9): 3.0, 3
NESU (8): 3.25, 3
DMDX (32): 3.38, 3.5
MatLab (5): 4.2, 4
Presentation (12): 4.54, 5
*
We also asked, "How easy/hard is it for a novice to learn how to build experiments with your software?" The mean difficulty rating was 4.12 with a standard deviation of 1.63, and a median of 4. Here are the mean and median build-difficulty ratings for the primarily used packages (for this analysis, PsyScope classic and PsyScope X were combined, because their response profiles were similar; SuperLab and SuperLab pro were different, so are reported separately):
SuperLab (5): 2.6, 3
PsyScope (29): 3.5, 3.5
E-Prime (57): 3.54, 3
ERTS (6): 4.17, 4.5
SuperLab Pro (4): 4.25, 4.5
DMDX (32): 4.88, 5
NESU (8): 5, 5.5
Presentation (12): 5.64, 6
MatLab (5): 6, 6
*
We then asked, "How satisfied are you with your current software?" with "1" meaning "Completely dissatisfied" and "7" meaning "Completely Satisfied." The mean satisfaction rating was 4.59 with a standard deviation of 1.43, and a median of 5. Here are the mean and median ratings by package:
DMDX (32): 5.09, 5
PsyScope X (11): 4.91, 5
E-Prime (57): 4.77, 5
MatLab (5): 4.4, 4
SuperLab (5): 4.2, 4
Presentation (12): 4.09, 4
NESU (8): 3.88, 4
PsyScope classic (18): 3.82, 4
SuperLab Pro (4): 3, 3
*
Two more quantitative questions. First, we asked "Are you sticking with your current software for the foreseeable future, or are you looking to change setups?" 25 respondents responded with "Don't Know," 103 with "Sticking with my current software for the foreseeable future," and 44 with "looking to change." Here's the breakdown by package, reporting percentages of those sticking with their package and those looking to change (sorted by sticking percentage):
PsyScope X (10): 90%, 10%
E-Prime (52): 71.1%, 9.6%
SuperLab (5): 60%, 40%
MatLab (5): 60%, 40%
DMDX (30): 53%, 20%
Presentation (11): 45.4%, 36.4%
ERTS (5): 40.0%, 60%
PsyScope classic (16): 37.5%, 62.5%
NESU (8): 12.5%, 62.5%
SuperLab Pro (4): 0%, 50%
*
Finally, we asked "Would you recommend your current software?" 138 people said "yes" and 32 said "no." Here's the breakdown of percent "yes" responses by package:
E-Prime (50): 92%
PsyScope X (10): 90%
PsyScope classic (16): 87.5%
DMDX (30): 86.7%
MatLab (5): 80%
Presentation (12): 63.6%
SuperLab (5): 60%
ERTS (5): 60%
SuperLab Pro (4): 50%
NESU (8): 50%
*
We also asked two complementary open-ended questions that aren't easy to summarize. One was, "What do you like about your current software? What are its strengths? What does it do well?" and the other was, "What do you not like about your current software? What are its weaknesses? What does it not do well (or at all)?" Considering the big hitters (E-Prime, DMDX, and PsyScope), my general impression of the flavor of the comments were:
E-Prime: Easy to learn, good support, user friendly, etc. But, some consider it expensive, thought it inflexible, and are worried about precision of timing.
DMDX: It's free, powerful, good timing, good user-support group, and good author support. Weaknesses were mostly regarding lack of intuitiveness and steep learning curve.
PsyScope: It's free, user-friendly, timing is accurate. But it can be buggy. PsyScope classic users are worried about using a legacy system. PsyScope X users worry about the transition to Intel-based Macs, but with some optimism.
*
Again, thanks for participating. We were thrilled to see that we actually had 187 people respond!
Best,
Vic Ferreira
Jeremy Boyd
Jeff Elman
Robert Buffington
Bob Slevc
Experiment Builder: http://www.eyelinkinfo.com/optns_eb.php
MEL: (Note that MEL is the predecessor to E-Prime)
UPDATE: Other relevant links (Thanks to Roberto Heredia):
UPDATE 2: Alan Garnham suggested to me that SuperLab and SuperLab Pro are not distinct products (though I swear I have a memory of their being something called ‘SuperLab’ without ‘SuperLab Pro’!). If I get a chance, I’ll combine the two in the above analyses.
source:
http://lpl.ucsd.edu/LabPage/Lab_Blog/B1A6A7D2-0069-41E3-89E9-B3683FEEC758.html
1 comment:
there is something wrong about the typing password script.
the line should be:
guessPin=input('Please input your PIN,type q to quit:','s')
or you need to type "'q'" to quit
Post a Comment