Sample selection process

Nele — Wed, 06/09/2010 - 14:09

~~~DRAFT~~~DRAFT~~~DRAFT~~~

Here I describe the criteria used to select the sample dojinshi and fanfics described in the data sets. All center on the character of Severus Snape.

Final list of dojinshi samples:

http://creator.zoho.com/nele.noppe/fanficforensics/#View:HP_dojinshi_dat...

Final list of fanfic samples:

http://creator.zoho.com/nele.noppe/fanficforensics/#View:HP_fanfic_data1

Dojinshi

Choosing dojinshi samples has been quite straightforward. There are far less dojinshi available than fanfics, and funds are needed to buy and transport them, so there was much less choice to begin with. Limiting samples by character and by publication date helped narrow the list down to the necessary 100 samples.

About ten forays into various dojinshi stores in Tokyo and Osaka in 2008 and 2009, plus several gifts of dojinshi, netted me between 250 and 300 stories that featured Severus Snape in some significant role. I tried to purchase every dojinshi I came across that seemed to feature the character, but since most dojinshi stores keep the books in sealed plastic bags, I had to go on covers alone.

I narrowed down this number considerably by eliminating all dojinshi written before the Japanese edition of 'Order of the Phoenix' went into print on September 1, 2004. (See http://creator.zoho.com/nele.noppe/fanficforensics/#View:Book_film_relea... for the release dates for HP books and films in Japanese and English) Compared to the previous four books, this installment of the Harry Potter series contained a substantial amount of back story on Severus Snape's canon relationships towards various other characters, most significantly his erstwhile schoolmates. This means dojinshika and fanfic writers who have read the fifth book have much more ready-made 'canon' material to work with, meaning that differences and similarities in the way they deal with this material will be all the more apparent.)

This left me with a little over 50 dojinshi. Since one dojinshi often contains several separate stories, and several of the 50 dojinshi selected are anthologies featuring stories by numerous different authors, I was left with virtually exactly 100 sample stories in January 2010.

(note: multiple dj by one author issue -discuss later)

Fanfics

Choosing fanfic samples was much more complicated, given the immense number of fanfics readily available online. In order to choose 100 more or less representative fanfics to compare with the 100 dojinshi, I needed to first determine which pairings were the most popular among HP fan creators. I did this by counting the number of LiveJournal users and communities who listed a certain pairing as an 'interest', and calculating relative percentages based on that. While LiveJournal-based HP fandom certainly cannot be said to be representative of all HP fan creators, I could not find a sensible way to compare pairing popularity across multiple fannish online communities and personal sites hosting HP fanwork. For this reason, I chose to focus on LiveJournal exclusively. Any conclusions drawn from the data should take this into account.

One important problem with the method of comparing 'interest' numbers is that there is no widely used 'interest' keyword for gen fanfics, meaning fanfics that do not feature any pairings between two characters. I ended up roughly estimating that 15 percent of fanfics featuring Snape in a main role were gen (see below).

I will detail the process of determining relative pairings popularity based on LiveJournal 'interests' below. Caveat: these numbers are a snapshot of the situation on January 3, 2010. The popularity of any given pairing changes over time, and communities and users may or may not alter their listed 'interests' as their tastes evolve. Also, communities and users often list their preferred pairings using several different keywords. For these reasons, these numbers can give only a rough approximation of the relative popularity of pairings over the past several years. Given that the fanfic and dojinshi samples used in the project have been written over a period of several years as well, I believe comparing listed interests in this manner is an appropriate way to gouge the relative popularity of fanfic pairings. Suggestions as to a more effective methodology are very welcome.

The process of determining relative pairings popularity in detail:

searched communities and users by interest via http://www.livejournal.com/interests.bml
keywords used: snarry, snupin, snaco, sshg, snape/harry, severus/harry, snape/lupin, severus/lupin, snape/hermione, snape/draco, and others (see data set for all keywords)
noted number of communities and number of users listing each keyword as an interest, in the dataset http://creator.zoho.com/nele.noppe/fanficforensics/#View:LiveJournal_int...
for each pairing, noted the keyword which was used by the most communities and users
using this most popular keyword per pairing as a touchstone, the pairings can be ranked as follows (most popular first):
1. Severus Snape and Harry Potter. Keyword: snarry. Communities: 145 Users: 458
2. Severus Snape and Remus Lupin. Keyword: snupin. Communities: 65 Users:436
3. Severus Snape and Hermione Granger. Keyword: snape/hermione. Communities: 59 Users: 423
4. Severus Snape and Lily Evans. Keyword: snape/lily. Communities: 49 Users: 268
5. Severus Snape and Draco Malfoy. Keyword: snape/draco. Communities: 29 Users: 138
6. Severus Snape and Lucius Malfoy. Keyword: snape/lucius. Communities: 27 Users: 107
7. Severus Snape and Sirius Black. Keyword: sirius/snape. Communities: 24 Users: 48

Harry Potter is clearly the character who enjoys the post popularity among Snape shippers. He is followed at a great distance by Remus Lupin, with Hermione Granger being a close third. Lily Evans follows at a short distance. Pairings including Draco Malfoy, Lucius Malfoy, or Sirius Black enjoy some measurable popularity, although these are often included in Snape 'rarepairs'. The popularity of other pairings seems limited enough to be statistically insignificant.

The full list of keywords searched and numbers of communities and users listing the keywords as an interest can be found at http://creator.zoho.com/nele.noppe/fanficforensics/#View:LiveJournal_int...

The process of choosing fanfic samples based on the abovementioned numbers:

Problem: no good way to find out what percentage of Snape-centric fics is gen (no easily searchable keywords, as with the pairing names)

On recs site Know-It-Alls (explain why this site), 15 percent of fics recommended is gen (on 30/01/2010, 352 gen fics recommended, 1067 slash, 1240 het). This is probably the best approximation we have, so 15 out of the 100 fics will be gen fics.

How to choose the 85 slash and het fics? Adding up the numbers of interested communities for each pairing mentioned above (using only the most popular keyword), we have 145+65+59+49+29+27+24=398. 398 is 85 percent of 468. 468-398=70. So, we may enter an estimate number of 70 communities for gen fics to the total of 398, and we can calculate the percentage each slash and het pairing occupies by calculating what percentage of 468 the pairing occupies.

Using the abovementioned method of calculation, we know the 100 samples should consist of:

Severus Snape and Harry Potter fics: 31
Severus Snape and Remus Lupin fics: 14
Severus Snape and Hermione Granger fics: 13
Severus Snape and Lily Evans fics: 10
Severus Snape and Draco Malfoy fics: 6
Severus Snape and Lucius Malfoy fics: 6
Severus Snape and Sirius Black fics: 5
Snape-centric gen fics: 15

As with dojinshi, I choose only fanfics published after the publication of OotP (June 21, 2003 in the case of the English-language edition -see http://creator.zoho.com/nele.noppe/fanficforensics/#View:Book_film_relea...). I attempted to divide the period between the publication of OotP and the present in three parts -between OotP and HBP, between HBP and DH, and post-DH- and choose as many fics from each period as there were dojinshi (adjusting for the fact that these periods are different for English-and Japanese-language fandom due to the lag in publication of the Japanese-language edition). Since there are approximately the same number of dojinshi for each of the three periods, it would be reasonable to choose about 33 fanfics for each period.

(With English-language fanfics, it's fairly easy to say when a fic is post-OotP, post-HBP, or post-DH. Since Japanese dojinshika may have heard of the new canon elements from each book before the Japanese edition came out, it's impossible to say with any amount of certainty if a dojinshi is post-OotP, post-HBP, or post-DH. For this reason, trying to calculate a precise number would not be very useful; since the numbers appear to point towards a roughly equal number of dojinshi from each period, we may as well go with that estimate.)

Limiting fanfic samples to fics centering on the character of Snape, published after OotP, on LiveJournal, still left me with an immensely large amount of relatively scattered potential samples. In order to further limit the number of samples and prevent my personal interests from influencing sample selection, I chose the required number of fics for each pairing, as well as the required number of gen fics, from recommended fanfics listed on the large HP fanfic recommendations site Know-It-Alls (http://mujaji.net/kia). Fanfics listed on a recs site may be said to be considered of fairly high 'quality'. I believe it is appropriate to select samples from these recs, as the dojinshi who end up being made and sold in stores are said to be on the higher end of the 'quality' spectrum as well.

I started at the oldest Know-It-Alls page for each pairing and worked my way up chronologically, choosing only publicly available fics hosted on LiveJournal, except in a few cases where the number of recs provided did not allow me to be so selective. In these cases, I selected fics not based on LiveJournal. I selected only fics that were not password-protected in any way and where a date of posting was clearly marked. After the 100 fics were selected, 2 turned out to be behind password protection after all, and I ended up replacing these with 2 other samples from the same time period and featuring the same pairing.

(note- is this complete enough?)