A design for sex-segregation research in T&T: A thought experiment

While I have many criticisms and concerns with the decision to pursue sex-segregation research in education in Trinidad and Tobago, especially in the top-down manner with which it is being imposed on schools, (I have outlined these elsewhere here and here)), I want to offer for discussion a research design that I consider to be potentially viable if one wished to pursue this line of action (i.e. cognizant of the limitations to the claims that can be made, the ‘ill-definedness’ of the variables/constructs themselves, the complexity and ethical difficulty of research with vulnerable social agents, and the difficulty of generalizability to other schools and of providing a sound basis upon which policy could be founded). I invite reasoned discussion and feedback on any aspect of the design from interested parties in the media and via email. As a scientist, one has to think through and actively construct a design that is fit to the research question, i.e. one that can potentially provide evidence to support or refute the claim. Research designs or experimental protocols do not magically materialize and aren’t simply thrown together from a tool-kit of parts. Ethically, one also has to consider potential harms and iatrogenic effects, again cognizant that one cannot know all possible benefits or potential harms before-hand. One of the frequently ignored or unspoken possible harmful effects of most intervention research in education is reduction in learning/performance, i.e. lower achievement is a possibility.

While I don’t believe that we can PROVE that sex-segregation ALONE improves performance – there are too many exogenous variables that influence the latter – I am assuming that very possibility, (mathematicians can do this: assume a converse or contradictory proposition), and ask, as a scientist, how could/would I go about showing that it did or did not? I am also assuming that researchers are not going to actually analyze blood, hormone and chromosomes to determine children’s sex. Such a protocol, while informative would be invasive and expensive on a large scale. They will rely on visible characteristics, socio-cultural conventions and the assignment given by parents as authorized by medical authorities at birth as a proxy or ‘good-enough’ indicator for biological sex. I am also assuming no questions will be asked about sexual identity, i.e. feeling like a man in a woman’s body etc., and heterosexuality will be assumed as these are not considered ‘relevant’ to the question by authorities in Trinidad and Tobago at this time. Sex will be assumed by looking at visible characteristics. Male and female would have to be defined as mutually exclusive categories in all regards relevant to the experiment. Whether this is a valid assumption for researchers or sound basis for policy is another matter. What will matter to policy-makers is not what sex students are in any objective way but what researchers and society say that they are for the purpose of the intervention.

To even have a chance at proving that sex-segregation alone improved performance nothing else would have to be changed, i.e. we’d have to keep the curriculum the same, the teacher, the assessment, there could be no sex-differentiated instruction etc. The design could be easily modified to test claims of the relationship between “sex-segregation + sex-differentiated instruction on performance.” But one could not determine if it were the segregation or the instruction and we would not have previously ruled out that it was segregation alone that was responsible. If baseline data was collected we’d have to know under what conditions and try to collect the experimental data under similar conditions. Given the networked nature of societies, and especially in a small place like Trinidad, interventions, observation and analysis would have to be on short time-frames initially to limit communication among individuals involved. While scientists may speculate, conjecture or theorize, they must not ‘overclaim’, i.e. their conclusions must be warranted from the evidence available. I do believe that small, very narrow claims are sustainable if the observations are about very small time frames initially where there is little possibility of individuals’ history affecting the observations and little time for outside influences to confound. We might for example be able to support/refute claims of the form, “based upon these definitions and assumptions about sex and educational outcomes, sex-segregation leads to greater/lesser/no change in outcome on this measure of this variable (retention/attention/performance etc) we deem important, of this idea/concept x from discipline y, under these learning conditions z for the period of time t.” Specificity is everything.

Here’s the first part of my design. Begin with a class of close to equal numbers of Males and Females matched as best as possible on other variables like real-age, developmental age, SES, etc. Then separate M/Fs for a single short period of Math, English, Chemistry, or whatever. Instruction is to be a constant, delivered by the same teacher, same lesson, same jokes etc. (In variations of this design I have a strategy for separating students that hopefully would not clue them in right away that they were being separated by sex, but it relies on a little deliberate deception.) The content and structure of the lesson would have to be well-designed prior to implementation via extensive consultation between researchers, subject matter specialists, and teachers taking into account what is known about teaching that concept in general and in our educational context in particular. I would recommend the concepts/skills chosen be something with which all students, boys and girls, have fairly equal difficulty and perhaps little, ideally no prior experience with in the curriculum at their age-level (but which is part of the curriculum). In this way the knowledge/skills that are being assessed are new and more importantly have little consequence for students in the long term, i.e. they are not part of a consequential exam. Examples abound in mathematics but in order not to confound the design if pursued I won’t reveal that aspect to the public just yet. Specifically, avoid concepts/tasks that are known to favour one sex over another like spatial rotation tasks or some verbally complex reasoning tasks. Ideally I’d want to see lessons drawn from all of the disciplines from Art to English, French to P.E. and Social Studies to Zoology as we are interested in sex differences across the curriculum, not just in Mathematics, Science, and English, though we might choose to begin there. The lessons and topics chosen should span a wide range of educational levels, from first-year (infants) to Form One to first-year of University. Again all lessons would need to be well designed, pedagogically and disciplinary sound and scripted before being used in an actual classroom.

The next part of my design is about delivery. In thinking about how to have students in experimental and control classes receive the same lesson from the same teacher at the same time given the resources available one approach is to use a standardized video lesson or virtual tutor. This could then be used at all schools on both islands with little cost beyond the initial production of the video and resources for showing, thus allowing for wider application to a broader sample, for example in single sex-schools. It would add a level of similarity and redundancy to the instruction at a reasonable cost and massive potential gain in knowledge. The video content should be short, 10-15 minutes. This considers that students will be aware that they don’t have the physical presence of a teacher and the authority represented present in the normal ways as well as the limitations of working memory and human attention. We could then make sound claims about our outcome measure(s) under these fairly fixed conditions thus avoiding the complications of different tutors and feedback at this early stage of the design. These important elements of instruction should be incorporated over time in other designs but not in the beginning. Different video lessons (within the same discipline) should involve teachers/tutors/actors that differ by gender, race, dress, accent etc. If well-designed these could be used for other research studies providing a larger return-on-investment.

If a class is separated the physical space and psychological space is changed. To compensate classes should be merged so that the total numbers remain the same else we are also testing possible effect of class size on the outcome phenomenon of interest. One group should not be sex-segregated but be kept intact and shown the same video. Another kept intact and not shown the video at all and yet another sex-segregated but not shown the same video (they should be shown a video of someone teaching something). That the content of the lessons are not regular curriculum topics at that level means that performance/non-performance on the measure(s) are not academically consequential for students’ lives in the short term. We wouldn’t want students to feel that they were being ranked/scored or valued in a way that could influence their future in this experiment. They get enough of that already.
We don’t necessarily have to show these videos in school settings, though it is likely to be more practical and economical to do so. In fact, we probably could make stronger claims by not doing so initially. Though we would lose the richness and complexity of the school classroom we would gain significantly by having a controlled environment in the beginning stages. Boys and girls could register or be randomly selected (by lottery for example), and volunteer to be part of this research and come to identical centres, located in different parts of either island, either after-school or on weekends, where they would be assigned to the various groups. I like this as it allows students to be simply boys and girls and not representatives of one school or another with all that social baggage. In fact groups could be constituted of students of relatively similar backgrounds from different types of schools if we thought this would yield interesting and relevant data. This would also reduce the influence of peer and cohort dynamics as everyone in a group might be relative strangers to each other. The issue here is whether or not some of the students we would like to be part of the research would actually be part of it, but the same applies in that the students we might be concerned about might not be in school on the day the research takes place. This is a serious problem, a significant sampling error and one for which I have not thought of a solution as yet that does not involve steps that I am not comfortable taking as a researcher. Incentives for participation in the research may be necessary.

The third part is about data-collection and analysis. The actions of students in the different groups as they engaged in the lesson should be recorded. Their talk, questions, movements, notes, doodles during and after the lesson and of course the assessment, form the initial data-set. I’m leaning more strongly now to fixed laboratory styled research settings to capture this data in the initial stages as described above and not existing schools. I am again assuming we’re not after physiological data such as cortisol, adrenaline or glucose levels (indicators of stress) or eye-movement (indicator of focus of attention) – at least not on a large scale. In the pre-stages of the video-lesson design eye-tracking studies might be important with a few students of different sexes to observe what they focus on before taking it to a classroom or experimental setting. One would look at actions during the same time across various groups, for example, in the first minute, or at fixed critical points in the lesson. Keeping the time restricted in this way allows for some strong claims to be made and backed-up.
Analysis of the classroom data should be supplemented with focus group or individual interviews about the lesson, learning, the sex-segregation dimension, performance, etc. As you can see this is a difficult and data rich design. We’re talking, at the minimum, many hours of video, thousands of pages of transcripts - terabytes of data - to be analyzed. A strong set of appropriate analytic resources from a wide range of fields should be brought to bear on the data including but not restricted to tools, concepts and methods drawn from fields as disparate as cognitive science, linguistics, gender studies, cultural studies, mathematics etc. Nobody said that research was fast, easy (or cheap).

A good research design is like a good story – well thought out, plausible plot and potentially satisfying to the consumer. More than this though the design I have offered above, while still in rough form, I think, offers many opportunities for new knowledge generation and innovation in education – one of the goals of this new government. Importantly the design is adaptable, transferable, and scalable. From these perspectives I believe such a project would be valuable. When I started writing this piece the design was meant to be implemented in schools. After receiving some feedback from more experienced peers, I am more convinced that it should first be pursued (or perhaps simultaneously pursued) via research laboratories, a network of knowledge-centres, set up across both islands in various communities where not only education but health or other inter and trans disciplinary knowledge projects could be pursued using the equipment there. This might be a better way to invest in innovation and capacity building for some types of research involving humans. I envision facilities that researchers from different disciplines or interested groups could book/rent with recording equipment already set up to capture voice, video, etc. [We’ll have to deal with crime and security but we have to do that anyway.] These could probably be added to or formed from already existing infrastructure eg. in community centre/health centres. Through such an initiative I see the possibility for involving more and more people in participating in the processes of knowledge creation through research and study in their own communities. Independent groups and entrepreneurs can likely begin to pursue this strategy without government subsidization but I would wish that they did so as social businesses instead of profit-making-businesses, i.e. not for any more individual profit than one’s initial investment. In my thinking here I am heavily indebted to the ideas of Dr. Mary King, Muhammad Yunis and the late Lloyd Best’s call for more research, auto-research in particular.

In recent months I have found the local educational conversations on sex-segregation to be superficial, stale, or stalled and not moving forward in a productive or meaningful way for the national community. Few have offered anything significant or meaningful from which the population could even begin to have a reasoned conversation. My hope with this piece is that by providing something substantial, albeit provisional, concerned and interested individuals will challenge, advise, direct and shape (or reject) this research design in a way that might be useful to us all through civil discourse. My goal in crowd-sourcing is both to get more people to engage in some aspects of scientific design thinking and to actually think deeply and rigorously about what would have to be done, and how that could happen, and ultimately why we want to do this and how to improve this research so that we get value for money and the answers we seek. I know that this is the right thing to do. It is an invitation that I sincerely hope is taken up.

In closing, I wish to remind readers of my earlier concerns about the definitions used and values underlying all research endeavors and this one in particular. These are not simple problems of epistemology and ontology that can be overlooked or left to philosophers but are often important determinants of the findings themselves. I have ignored them only to be able to begin to speak to those who are not familiar with those discourses. One learns a great deal from thinking, reading, observation and experiment, but it matters a great deal with what tools and which theoretical frameworks one chooses to use and how one chooses to wield them. Not all frameworks or designs (or writers) however are mindful of their capacity for harm, respectful of the dignity of the diversity of all (human) research participants or honors the integrity of self-determining beings. Such considerations though must be among the very first in any research programme.

The Reader Bytes Back

“The ethics of criticism requires pointing out the faults in a colleague's thinking.” (Jeffrey R. Di Lio, 2010)

In the digital era one of the tasks facing teachers of the art of persuasive writing is the responsible use of source material. There are many dimensions to responsible use. One of these is appropriate attribution of sources, the breach of which, in an argumentative setting by one seeking to perform as a scholar or intellectual, public, or otherwise, is plagiarism, a species of academic dishonesty. This is a lesson Fr. Henry Charles, President Richards and T&T will likely not forget any time soon. A more difficult skill for many argumentative writers, one exacerbated by the hyper-geometric increase in available and accessible information is responsible use of source material. Irresponsible use of sources, in the context of persuasive writing or scholarly work, takes many forms ranging from the professed naïveté of students who print out Wikipedia articles or web-pages, place them in folders and try to pass them off as ‘research’ papers, to more subtle scholarly improprieties such as that employed by Kevin Baldeosingh in his article Laptops dance with danger (11/06/10).

Baldeosingh’s arguments are typically presented as an intertextual mashup – a sampling of exemplary excerpts from various commentaries made in public fora which are eviscerated with sardonic commentary while buttressing his own position by appealing to ‘expert’ testimony drawn from one or more of his stable of scientists du jour. His style is, usually, a hybrid genre blending acerbic satire and critique and last Friday’s is no exception. Now satire, when it is done well, can be an effective means of critique, persuasion and entertainment, and there have been occasions when he has achieved this harmony to great effect. Last week’s piece however is neither effective nor well done and bad satire is neither an excuse for, nor can it serve as a rhetorical sanctuary from professional irresponsibility as a writer and journalist.

The article represents an example of irresponsible use of source material as Baldeosingh excises and presents segments of text from the letters of Drs. Sharma and Kalicharan as metonymies for positions they do not advocate. Severed from the contexts of their original utterance, Baldeosingh’s injudicious juxtapositions facilitate the construction of these individuals’ contributions to the national conversation on education as simplistic, spurious and suspicious. In short, he misrepresents their arguments through selective quotation in order to generate (false) ‘evidence’ of the antagonistic position he wishes to critique. He invents his opponents.

Consider the way Baldeosingh very early on elevates the emotional ante by descriptively framing the contributions as “dire warnings” and “passionate denunciations.” Are they really? As a scholar when one suspects that what one is reading is untrustworthy or that claims are exaggerated, or worse unfounded, it is incumbent to check for oneself, and indeed a laptop and internet connection makes this easier than ever. I mean no offense to Dr. Sharma, but in her writing she is rarely what I would call a ‘passionate denunciator’ of any kind. She is typically measured, and restrained. This is what aroused my suspicions and led me back to the sources – the three public commentaries.

I noticed - and readers are advised to check this for themselves - that Sharma offered neither dire warnings nor passionate denunciations. Rather, she attempted to raise critical “issues that need to be considered to honour this commitment” from her perspective as an educator who works with teachers across Trinidad and Tobago. She does not isolate the potential benefits from the possible iatrogenic effects and the practical difficulties likely to be encountered in implementing this election promise that voters forget is not yet educational policy. She wisely situates the decision within the culture and climate of schools in T&T where violence, indiscipline, bullying, theft, infrastructural, human and knowledge resource deficiencies, and a lack of training and supervision are commonplace. In short she does not over-simplify the matter but attempts to present and retain some of the complexity of the situation. Likewise, reading Kalicharan’s letter for myself I take it as a public intellectual’s wondering and questioning of the rationale behind this decision to give laptops away. His central point, like Sharma’s, is that laptops alone are not going to be a panacea for some of the challenges facing education in Trinidad and Tobago. Baldeosingh eschews these central points completely in his attempts to paint these educators as alarmists though it is he who lays the foundation for a media-sustained panic discourse through his interpretation and public presentation of their concerns as “dire warnings” and “passionate denunciations.”

There is some evidence of Baldeosingh’s success in at least one case that I could find and which serves as an example of why I consider this particular piece of writing irresponsible. Consider, Trinizagada, a cotecicotelatnt blogger’s response to this article, “After reading the article (Baldeosingh’s) in the Trinidad Express I was lost for words. Now the laptop program of the People's Partnership is going to create morons? What in heavens name are these so called educated people thinking?” S/he goes on to describe details about a laptop programme in the US. To be fair, many in T&T do not read satire well. We respond to picong, brutish humiliation and humbling of authorities and experts when they can be shown to be talking/doing nonsense, and we are fortunate to have an unnatural abundance so as not to have to invent such opportunities, but that is not the case here and is certainly not my intent with this critique.

By irresponsibly misrepresenting and misreporting the central concerns raised by Sharma and Kalicharan, Baldeosingh has influenced this blogger’s, and perhaps some of the general public’s, opinion in a way that undermines their credibility as educators. This is likely not libelous as the piece is sufficiently satirical for this argument to function as a valid defense. In an academic setting however the student would be counseled to return to the sources, read them more carefully, attend to what they are actually saying and responsibly re-present their positions before critiquing them. Indeed, I note that of the three writers (Sharma, Kalicharan, Furlonge), the only person who actually constructs a link between laptops, reduced intelligence and moral questionability (morons, thieves, perverts, politicians, priests, pundits) is Baldeosingh himself. He manages however to successfully obscure this fact by creating and transferring a set of false associations with these ideas and those of the letter-writers. That deception finds any home in Baldeosingh’s argumentative algorithm calls into question his corpus of work as a reliable writer and commentator as well as his ethics. There are other examples in Baldeosingh’s piece, but taking a cue from many good math texts, and a proven strategy for prompting ‘active learning’, “the remainder of the proof is left as an exercise for the interested reader.”

Perhaps of greater concern is the way Baldeosingh deliberately uses misdirection and scientific language to fashion an identity as a knowledge-broker. Globally, there is great concern about the public’s lack of critical understanding of the science, and scientific thinking necessary for enacting democratic citizenship and in helping societies to govern themselves. News media are tasked, perhaps unfairly, with an important educative role in this project. Baldeosingh’s work this week, atypically, works against such goals. In presenting the evidence for his own case Baldeosingh also oversimplifies and misrepresents his sources. Citing psychologist Richard Nesbitt and journalist Steven Johnson he argues for the importance of play in learning and the contributions of popular culture, video-games in particular via Nesbitt, to increasing IQ scores. Baldeosingh neglects however the importance of context to learning and the limits of such learning, viz. the difficulty of transfer. Fortunately, Harvard psychology professor Steven Pinker, (someone who also apparently doesn’t teach) provides the missing nuance. In a New York Times Op-ed piece which appeared on the same day as Baldeosingh’s article, he notes that, “If you train people to do one thing (recognize shapes, solve math puzzles, find hidden words), they get better at doing that thing, but almost nothing else…Accomplished people…immerse themselves in their fields.” Despite many of my former students’ protestations, playing FIFA’s video games will only make you a better video-game player, but it won’t get you to a World Cup or improve your skills significantly on the field without significant physical, emotional and mental effort. I don’t think Farmville is going to help us solve our food problems.

Pinker also makes the important educational point that mere access to technology is not going to suddenly usher in some democratic educational utopia. I have made similar arguments in my scholarly publication Harnessing the Complexity of Children’s Consumer Culture, in particular I believe that while popular culture is a powerful pedagogue, there are some things it does not teach very well and indeed is inimical to the development of some important human and democratic values. Pinker likewise argues that good and responsible intellectual habits “of deep reflection, thorough research and rigorous reasoning” have never been easy to develop, rather, “they must be acquired in special institutions…and maintained with constant upkeep, which we call analysis, criticism and debate. They are not granted by propping a heavy encyclopedia on your lap, nor are they taken away by efficient access to information on the Internet.” Imagining what those institutions might be like, understanding why they have found difficulty flourishing locally and then working together to actively construct ones that will work for us to achieve the goals of good, responsible habits of mind and being are what we ought to be debating. These are but a few of the tasks which the privileged people who “can’t teach” should be doing and doing well. It is partly what I have tried to do here.

Finally, Baldeosingh notes rhetorically, “when the lower orders begin to get smarter, they aren’t as easy to fool, which leads to all kinds of dangerous trends…do we really want a society where students challenge teachers…?” The same resources that destabilize the authority of the professor and the priests and their claim to being the final, irrefutable source and arbiters of knowledge can also be applied to other knowledge producing agents including media practitioners and cultural workers but we must be care full. As I continue to advocate we must learn to read and write different types of texts more carefully and critically than we have been doing. We must also teach these skills and be prepared for the painful occasions when they will be applied to our own works.

If democracy and skepticism, digital and otherwise, are to flourish in T&T it will not likely be without effort, struggle, and the courage to take on powerful gatekeepers, including journalists, when they act irresponsibly without also acting irresponsibly ourselves. If we can learn that responsible critique and debate do not necessarily have to humiliate, brutalise, or belittle others to be effective, educative or entertaining, and that compassion is not an estranged kin of argument, then perhaps we might create conditions for fecund conversations. If we can demonstrate that we are a society where thoughtful readers can challenge established writers to be first ‘better’, i.e. more conscientious, in their craft, and secondly, better human beings, then perhaps we might truly create new, more fit models of dialogue for our civilization…