A design for sex-segregation research in T&T: A thought experiment

While I have many criticisms and concerns with the decision to pursue sex-segregation research in education in Trinidad and Tobago, especially in the top-down manner with which it is being imposed on schools, (I have outlined these elsewhere here and here)), I want to offer for discussion a research design that I consider to be potentially viable if one wished to pursue this line of action (i.e. cognizant of the limitations to the claims that can be made, the ‘ill-definedness’ of the variables/constructs themselves, the complexity and ethical difficulty of research with vulnerable social agents, and the difficulty of generalizability to other schools and of providing a sound basis upon which policy could be founded). I invite reasoned discussion and feedback on any aspect of the design from interested parties in the media and via email. As a scientist, one has to think through and actively construct a design that is fit to the research question, i.e. one that can potentially provide evidence to support or refute the claim. Research designs or experimental protocols do not magically materialize and aren’t simply thrown together from a tool-kit of parts. Ethically, one also has to consider potential harms and iatrogenic effects, again cognizant that one cannot know all possible benefits or potential harms before-hand. One of the frequently ignored or unspoken possible harmful effects of most intervention research in education is reduction in learning/performance, i.e. lower achievement is a possibility.

While I don’t believe that we can PROVE that sex-segregation ALONE improves performance – there are too many exogenous variables that influence the latter – I am assuming that very possibility, (mathematicians can do this: assume a converse or contradictory proposition), and ask, as a scientist, how could/would I go about showing that it did or did not? I am also assuming that researchers are not going to actually analyze blood, hormone and chromosomes to determine children’s sex. Such a protocol, while informative would be invasive and expensive on a large scale. They will rely on visible characteristics, socio-cultural conventions and the assignment given by parents as authorized by medical authorities at birth as a proxy or ‘good-enough’ indicator for biological sex. I am also assuming no questions will be asked about sexual identity, i.e. feeling like a man in a woman’s body etc., and heterosexuality will be assumed as these are not considered ‘relevant’ to the question by authorities in Trinidad and Tobago at this time. Sex will be assumed by looking at visible characteristics. Male and female would have to be defined as mutually exclusive categories in all regards relevant to the experiment. Whether this is a valid assumption for researchers or sound basis for policy is another matter. What will matter to policy-makers is not what sex students are in any objective way but what researchers and society say that they are for the purpose of the intervention.

To even have a chance at proving that sex-segregation alone improved performance nothing else would have to be changed, i.e. we’d have to keep the curriculum the same, the teacher, the assessment, there could be no sex-differentiated instruction etc. The design could be easily modified to test claims of the relationship between “sex-segregation + sex-differentiated instruction on performance.” But one could not determine if it were the segregation or the instruction and we would not have previously ruled out that it was segregation alone that was responsible. If baseline data was collected we’d have to know under what conditions and try to collect the experimental data under similar conditions. Given the networked nature of societies, and especially in a small place like Trinidad, interventions, observation and analysis would have to be on short time-frames initially to limit communication among individuals involved. While scientists may speculate, conjecture or theorize, they must not ‘overclaim’, i.e. their conclusions must be warranted from the evidence available. I do believe that small, very narrow claims are sustainable if the observations are about very small time frames initially where there is little possibility of individuals’ history affecting the observations and little time for outside influences to confound. We might for example be able to support/refute claims of the form, “based upon these definitions and assumptions about sex and educational outcomes, sex-segregation leads to greater/lesser/no change in outcome on this measure of this variable (retention/attention/performance etc) we deem important, of this idea/concept x from discipline y, under these learning conditions z for the period of time t.” Specificity is everything.

Here’s the first part of my design. Begin with a class of close to equal numbers of Males and Females matched as best as possible on other variables like real-age, developmental age, SES, etc. Then separate M/Fs for a single short period of Math, English, Chemistry, or whatever. Instruction is to be a constant, delivered by the same teacher, same lesson, same jokes etc. (In variations of this design I have a strategy for separating students that hopefully would not clue them in right away that they were being separated by sex, but it relies on a little deliberate deception.) The content and structure of the lesson would have to be well-designed prior to implementation via extensive consultation between researchers, subject matter specialists, and teachers taking into account what is known about teaching that concept in general and in our educational context in particular. I would recommend the concepts/skills chosen be something with which all students, boys and girls, have fairly equal difficulty and perhaps little, ideally no prior experience with in the curriculum at their age-level (but which is part of the curriculum). In this way the knowledge/skills that are being assessed are new and more importantly have little consequence for students in the long term, i.e. they are not part of a consequential exam. Examples abound in mathematics but in order not to confound the design if pursued I won’t reveal that aspect to the public just yet. Specifically, avoid concepts/tasks that are known to favour one sex over another like spatial rotation tasks or some verbally complex reasoning tasks. Ideally I’d want to see lessons drawn from all of the disciplines from Art to English, French to P.E. and Social Studies to Zoology as we are interested in sex differences across the curriculum, not just in Mathematics, Science, and English, though we might choose to begin there. The lessons and topics chosen should span a wide range of educational levels, from first-year (infants) to Form One to first-year of University. Again all lessons would need to be well designed, pedagogically and disciplinary sound and scripted before being used in an actual classroom.

The next part of my design is about delivery. In thinking about how to have students in experimental and control classes receive the same lesson from the same teacher at the same time given the resources available one approach is to use a standardized video lesson or virtual tutor. This could then be used at all schools on both islands with little cost beyond the initial production of the video and resources for showing, thus allowing for wider application to a broader sample, for example in single sex-schools. It would add a level of similarity and redundancy to the instruction at a reasonable cost and massive potential gain in knowledge. The video content should be short, 10-15 minutes. This considers that students will be aware that they don’t have the physical presence of a teacher and the authority represented present in the normal ways as well as the limitations of working memory and human attention. We could then make sound claims about our outcome measure(s) under these fairly fixed conditions thus avoiding the complications of different tutors and feedback at this early stage of the design. These important elements of instruction should be incorporated over time in other designs but not in the beginning. Different video lessons (within the same discipline) should involve teachers/tutors/actors that differ by gender, race, dress, accent etc. If well-designed these could be used for other research studies providing a larger return-on-investment.

If a class is separated the physical space and psychological space is changed. To compensate classes should be merged so that the total numbers remain the same else we are also testing possible effect of class size on the outcome phenomenon of interest. One group should not be sex-segregated but be kept intact and shown the same video. Another kept intact and not shown the video at all and yet another sex-segregated but not shown the same video (they should be shown a video of someone teaching something). That the content of the lessons are not regular curriculum topics at that level means that performance/non-performance on the measure(s) are not academically consequential for students’ lives in the short term. We wouldn’t want students to feel that they were being ranked/scored or valued in a way that could influence their future in this experiment. They get enough of that already.
We don’t necessarily have to show these videos in school settings, though it is likely to be more practical and economical to do so. In fact, we probably could make stronger claims by not doing so initially. Though we would lose the richness and complexity of the school classroom we would gain significantly by having a controlled environment in the beginning stages. Boys and girls could register or be randomly selected (by lottery for example), and volunteer to be part of this research and come to identical centres, located in different parts of either island, either after-school or on weekends, where they would be assigned to the various groups. I like this as it allows students to be simply boys and girls and not representatives of one school or another with all that social baggage. In fact groups could be constituted of students of relatively similar backgrounds from different types of schools if we thought this would yield interesting and relevant data. This would also reduce the influence of peer and cohort dynamics as everyone in a group might be relative strangers to each other. The issue here is whether or not some of the students we would like to be part of the research would actually be part of it, but the same applies in that the students we might be concerned about might not be in school on the day the research takes place. This is a serious problem, a significant sampling error and one for which I have not thought of a solution as yet that does not involve steps that I am not comfortable taking as a researcher. Incentives for participation in the research may be necessary.

The third part is about data-collection and analysis. The actions of students in the different groups as they engaged in the lesson should be recorded. Their talk, questions, movements, notes, doodles during and after the lesson and of course the assessment, form the initial data-set. I’m leaning more strongly now to fixed laboratory styled research settings to capture this data in the initial stages as described above and not existing schools. I am again assuming we’re not after physiological data such as cortisol, adrenaline or glucose levels (indicators of stress) or eye-movement (indicator of focus of attention) – at least not on a large scale. In the pre-stages of the video-lesson design eye-tracking studies might be important with a few students of different sexes to observe what they focus on before taking it to a classroom or experimental setting. One would look at actions during the same time across various groups, for example, in the first minute, or at fixed critical points in the lesson. Keeping the time restricted in this way allows for some strong claims to be made and backed-up.
Analysis of the classroom data should be supplemented with focus group or individual interviews about the lesson, learning, the sex-segregation dimension, performance, etc. As you can see this is a difficult and data rich design. We’re talking, at the minimum, many hours of video, thousands of pages of transcripts - terabytes of data - to be analyzed. A strong set of appropriate analytic resources from a wide range of fields should be brought to bear on the data including but not restricted to tools, concepts and methods drawn from fields as disparate as cognitive science, linguistics, gender studies, cultural studies, mathematics etc. Nobody said that research was fast, easy (or cheap).

A good research design is like a good story – well thought out, plausible plot and potentially satisfying to the consumer. More than this though the design I have offered above, while still in rough form, I think, offers many opportunities for new knowledge generation and innovation in education – one of the goals of this new government. Importantly the design is adaptable, transferable, and scalable. From these perspectives I believe such a project would be valuable. When I started writing this piece the design was meant to be implemented in schools. After receiving some feedback from more experienced peers, I am more convinced that it should first be pursued (or perhaps simultaneously pursued) via research laboratories, a network of knowledge-centres, set up across both islands in various communities where not only education but health or other inter and trans disciplinary knowledge projects could be pursued using the equipment there. This might be a better way to invest in innovation and capacity building for some types of research involving humans. I envision facilities that researchers from different disciplines or interested groups could book/rent with recording equipment already set up to capture voice, video, etc. [We’ll have to deal with crime and security but we have to do that anyway.] These could probably be added to or formed from already existing infrastructure eg. in community centre/health centres. Through such an initiative I see the possibility for involving more and more people in participating in the processes of knowledge creation through research and study in their own communities. Independent groups and entrepreneurs can likely begin to pursue this strategy without government subsidization but I would wish that they did so as social businesses instead of profit-making-businesses, i.e. not for any more individual profit than one’s initial investment. In my thinking here I am heavily indebted to the ideas of Dr. Mary King, Muhammad Yunis and the late Lloyd Best’s call for more research, auto-research in particular.

In recent months I have found the local educational conversations on sex-segregation to be superficial, stale, or stalled and not moving forward in a productive or meaningful way for the national community. Few have offered anything significant or meaningful from which the population could even begin to have a reasoned conversation. My hope with this piece is that by providing something substantial, albeit provisional, concerned and interested individuals will challenge, advise, direct and shape (or reject) this research design in a way that might be useful to us all through civil discourse. My goal in crowd-sourcing is both to get more people to engage in some aspects of scientific design thinking and to actually think deeply and rigorously about what would have to be done, and how that could happen, and ultimately why we want to do this and how to improve this research so that we get value for money and the answers we seek. I know that this is the right thing to do. It is an invitation that I sincerely hope is taken up.

In closing, I wish to remind readers of my earlier concerns about the definitions used and values underlying all research endeavors and this one in particular. These are not simple problems of epistemology and ontology that can be overlooked or left to philosophers but are often important determinants of the findings themselves. I have ignored them only to be able to begin to speak to those who are not familiar with those discourses. One learns a great deal from thinking, reading, observation and experiment, but it matters a great deal with what tools and which theoretical frameworks one chooses to use and how one chooses to wield them. Not all frameworks or designs (or writers) however are mindful of their capacity for harm, respectful of the dignity of the diversity of all (human) research participants or honors the integrity of self-determining beings. Such considerations though must be among the very first in any research programme.