Difficulty ratings for lessons

Yes, this would be very helpful! i also spent much time in finding "beginner Intermediate" lessons...

@buschstadium310: You can get all the podcasts for free, you just have to pay extra if you want things like Expansion and PDFs of the dialogs. Rosetta Stone is double the price of a premium CPod subscription.

As I have said before, I am all for it. Only we should arrive at some system that measures this level of difficulty objectively. I have an idea for a procedure, that would involve some automated measurement. The key would be the ratio of new words to know words in the lesson. This would be a way to achieve this rating: The basic idea would be that for each level there is a set of vocabulary that is the basic requirement for this level. In order to find out what this basic vocabulary is all the lessons for that level should be given over to a parser and subsequently a distribution table should be made of this complete vocabulary. Supose we do this for the 250 lessons of so of the Newbie level. Then we would end up with a table containing say 800 different words (consisting each of one or more characters) and the number of times that each word occurs in all of these lessons. The next step would be to determine a cut-off point for this level. This would be a matter of debate and would actually depend on the wordlist itself, but say we cut it off at 400 words. This would give us the list of the "core vocabulary" at Newbie level. Now we could use this list as a yardstick to measure each lesson. The vocab of each individual lesson should be compare to this core vocab. The ratio of 'number of words not in the list' to 'number of words in the list' can then be determined for each lesson. If the ratio is high, then the lesson diffucult, if the ratio is low, it should be considered easy, with 0 being the most easy. Now we could leave the ratio as it is, but it would be better to look at the distribution for these ratios for a number of lessons and decide on ranges that would subsequently correspond to discrete difficulty levels, say 1 up to 5. The advantage of this method is that it gives difficulty level independently of the individual student level, and that it grades the lessons individually but within their group of Newbie, Elementary, etc. This is the idea I have been working on. I would program it in Java, the parser is available. The coding is not too difficult. There would be some work involved of course. Especially in building the core vocab list for every level. But I think that once this is established it will not have to be changed a lot. Maybe once every year or so, to take into account the new lessons. Maybe I will find the time to try my hand at this. But that would not be before summer at the earliest. Marc

Marc: I can see that your method might be a more scientific way to calculate difficulty, but it would be quite difficult to decide which words are Newbie, which are Elementary, etc. Also, an Elementary lesson may contain nearly all Newbie words, but contain several difficult grammar constructions - a computer program would rate it as easy, but it may actually be hard. It's still a good idea though :) . I hope CPod picks up on this thread.

I also see some merit in Marc's idea of ranking languages by vocabulary. Its strength is that it uses a measure easily tallied by machine (i.e. characters). One would have to be careful however. Often a Cpod lesson's difficulty is determined by an individual's familiarity with the subject matter. Since lessons often cover narrow areas (e.g. apartment hunting), what is difficult for one person might be easy for another. I feel this becomes especially pronounced at the Intermediate level and above. Personally, I doubt that I will use such a ranking. Usually, if I'm having difficulty with a lesson, I just accept that I won't understand everything and move on. However, your idea is one of the more thoughtful ones I've heard. If others find it useful...

Of course a mechanical way of rating could never be perfect, put it has the advantage that it can be applied automatically. A way around could be to use subjective factors too, like grammar and subkect matter and to give them a weighting on top of the automatic one. So far, my method is just an idea. I will follow it up though, like I said, it is just a matter of time that I don't have for the moment. The programming side of it, is not even that difficult. But I will keep you all informed if it leads to something.

Sorry, but I think its a lot of work for little benefit. I, like Tvan, take them as they come and if difficult, accept that maybe I wont get all of it and move on. Months later return to that lesson and you will find it different.

From any reasonably practical standpoint, I don't think it's realistically possible to do this. Trying to come up with some 'objective' measure wold be, I think, too difficult and not yield any significant benefit. And to try anything subjective? What's "difficult"? THAT word is difficult for ME. Is it for everyone? The first few lessons I reviewed were 'harder' just because I was totally new. After I've gone through a few do I go back and re-rate them using my current mindset? When (IF, for me! :) ) I move from Newbie to Elementary then I'll just be in the same situation again. Listen to a lesson. If it feels too difficult I go on to another. As I build up vocab and a better 'feel' I can go back to the 'harder' lessons. Not perfect, but... I would, possibly, like to see a way of marking lessons I've covered that I don't want to go back to, those that I do want to review again and those I haven't seen. (My memory fades!) Still, good thoughts, though, and maybe something I don't see may be really workable. Keep up all the good work and I'm glad to see constant thoughts and ideas about improving (might be hard!) this learning process.

Oops, guess RJ was posting more quickly - and more succinctly - the same thought as me.

Steve, What you are asking for is already there. You can bookmark a lesson, 'remove' it or mark it as 'studied'. Your personal archive will only show the bookmarked lessons. As for the tactic of moving on to a lesson that suits you. Yes, but it takes time to assess that. For me the situation was the following: I would listen to an elementary lesson and understand the dialogue right away, maybe with the exception of 1 or 2 words. Reviewing the lesson, I could write all the characters. The exercises: no problem. The expansion section: maybe 3-5 new words and a few new characters. So I figured out I was ready for Intermediate.....not so, as most intermediate lessons were just beyond me. Except for a few simple phrases I could not understand anything from the dialogues, even after listening to these for 5-6 times. Every lesson would maybe have 30 new words and 15-20 new characters. About 50% of the sentences of the expansion section were beyond me. So I felt lost, and the whole CP adventure became very boring. My point is that at this moment in time there is a gap and that even tough a few recent lessons have become easier, nothing much is done to take the poor student in this transitional phase by the hand and lead him throught these murky waters. Now for a commercial company this makes little sense because I am convinced that this leads to a high drop out ratio. Without some other resources and some strong emotional and practical support by my Chinese internet friends, I would have dropped out of CP. And I have been paying since 2006! So my advice is to do something about this. Please CP,for your own sake, address this issue. Rating the lessons could be an element in this. Providing a "lower intermediate level" could be another element. (after all there is an 'upper intermediate', so why not have a 'lower intermediate'). Another possibility would be to introduce short series of transitional lessons. To take someone from one level to the next.

Yes, good idea.

RJ, I understand what you are saying man...I think I put in the work. I study 2 hours a day, 7 days a week. And I maintain a correspondence which is 20% chinese and 80% english with some Chinese friends. I am concerned that is all because the ride isn't as smooth as it should be. Some of the lessons at intermediate level are ok. Some are very difficult and very long. And it is hard to tell which one is right for you unless you have spent say 15-20 minutes on it. And even then. I am just making this point because Ken is such a fan of Stephen Krashen (http://www.sk.com.br/sk-krash.html) of "input + 1" fame. The + 1 that is supposed to facilitate language acquisition is not respected with the situation as is between the elementary and intermediate levels. For me it was more like input + 5 or so. So, effectively, what I had to do was to look out for other sources to provide me with the appropiate material, giving me the appropiate stimulus of i + 1 until I reached the CPod intermediate level. Maybe I am stubborn in all this, but I think that it is possible to come up with some sort of ranking that could help people to pick the right lessons first. And I think it should be vocab based for at least 80%. The idea would be that a 'level x newcomer' could plan his program according to indicated rank/grade and subject interest. After having worked through all the lessons ranked difficulty level 1 that are of interest, this student could then tackle difficulty level 2, etc. Especially now that there are so many lessons to choose from, this could be a great help to bring sone structure in the lessons offered. Contrast this with the current situation: People pick lessons at random. Often favouring the latest ones, but I know of some people who work systematically from beginning to end in a certain level. What happens is that they often fall onto lessons that are simply way too difficult for their current level. Now, we are all studying on our own, so motivation is essentially self-motivation. It is not the same as being in a class. We need a gentle path.. In the mean time, I have another plan to come up with some sort of rating. Probably easier to implement too. I think I may run some tests in a few weeks. If something meaningfull comes out of this, I will keep you all informed. Marc

March - looks like overkill to me, but one problem is that different people have different elementary vocab. If you drove the lesson rating off an individual vocab list however it might make more sense. There is no objective measure as such because different individuals have different vocabs.

Can't Henning's volume of work on intermediate lesson grammar points help in some way? I'd really like to be able to able to access that more easily. I think it is not being put to good use. Where the wheels come off for me are the expansion exercises. I do appreciate being exposed to other ways of using the vocabulary in the lesson, but I when I have 60 new words to learn in the expansion exercise, it really doesn't help me to expand my understanding of the lesson vocabulary. Even if I find the intermediate lesson relatively "easy", the expansion is usually just too much. I admire anyone who can actually say those sentences in "the fix" after studying the lesson and the expansion. They are way above my level even though I study intermediate lessons. I keep thinking that one day I'll get better and they will get easier...hasn't happened yet.

Dear Marc (marchey), thank you for that information about Stephen Krashen, which opens a window into the minds behind CPOD. For what it's worth, I don't think you're being "stubborn". Using Krashen's own language, it looks to me like "input+1" depends on finding a good balance between "acquisition" and "learning". "Acquisition" may be more important, but that does not mean that "learning" is not essential. Skimming through Krashen's "big" paper to acquaint myself with his ideas, I was struck in particular by his interpretation of first language interference. Eg. An English speaker who writes or speaks Chinese that is ordered by English, rather than Chinese, syntax. If I'm not mistaken, Krashen suggests that such interference may point to "low acquisition". He even compares it with the "silent period" when infants are acquiring language (basically, absorbing it from their enviroment), and goes so far as to suggest that learners -- both adults and infants -- in this "silent period" might not benefit from being encouraged to speak before their acquisition has reached a certain level. My understanding is that Krashen is saying they would be better off allowing their "acquisition" of language sufficient time to take its own natural course before putting it to work. Although I don't really agree with Krashen on this particular point, neither can I avoid being "niggled" by the idea that a student of Mandarin could (conceivably) develop speech habits which will be strongly marked by interference from the first language. Maybe there is a healthy balance between mindless drilling/ rote learning, and "Mandarin on Your Terms (taken to the extremes)"? CPOD seems to be doing a fairly good job in finding this balance. So many things that we do in life feel so natural with a bit of practice -- eg. using a stick shift, cutting up a whole chicken, using chopsticks, chopping onions quickly "TV chef"-style, swinging a golf club etc etc -- that I can't helping thinking that there is something to be said for a small amount of intelligent "drilling" in Mandarin.

Going back to Rsmith91's original post, why not have the users give each lesson a difficulty rank? Would be much, much easier to implement than the alternatives. I've always viewed the "diary" serials almost as upper-elementary lesssons, due to their length and no repetitions of the dialogue or translations. If given the option of granting difficulty stars, I'd probably give those 5 out of 5 (compared to the other elementary lessons). If such a system were already in place, I'm sure many of us would fare better starting off at 1 or 2 star intermediate lessons (which must exist, though there's no way to identify them) rather than being bashed across the head with something we're simply unprepared for.

I like mandomikey's example of how lessons can be hard or easy. That's the sort of thing I was getting at in my original post. :) There's been a lot of talk on this page about having a computer rate lessons for difficulty. Sorry, but I'm really not convinced that it would work. Aside from all the hard work that would be necessary, there's a huge problem: language is something very human, not like science or math that can be programed into a computer. Language is not just about vocabulary - it's about grammar, style and loads of other things that a computer can't understand. I think real humans would be able to rate lessons more accurately. A potential flaw with my plan that has been brought up is that someone who has only just moved onto a certain level may rate all those lessons as really hard, whereas someone who has been studying higher level lessons would rate it really easy. I don't see that this would be a problem - OK, it would happen now and again, but it should average out And I'd hope that people would have the common sense not to rate a lesson if it's the first one they've ever done from a particular level. Thanks for all the feedback everyone!

whats up with having to pay?? i read about you guys in a magazine and it said you were 100% free!! i might as well buy Rosetta Stone @ the price you guys are charging!!

I'm back on Cpod after a one year hiatus only to find the site completely different than when I first stumbled on it. Some of those changes have been for the better, others--not so much. This topic of establishing a difficulty rating is something that should be given its due and proper. Another tool that might help is to have the audio from the lesson vocab read back to you at different speeds. A newbie like me still isn't acquainted enough with the four tones to decipher and commit to memory the words that are being read back at blazing speeds. Example, Lesson titled "Call Me Back". The podcast is quite discernable but going through the vocab section and playing the audio clips is not. Can a rating system be implemented here? Can there be a newbie, interm. and advanced listening choice?

Marchey, I never could understand all the fuss when the pdf file is provided. Are we talking about those that dont have premium memberships or you are saying the pdf file is not enough help? Are you complaining about the dialog only between Jen and John? Chinese is difficult. What can I say. Man up a little. :-) Intermediate evidently is the level that separates the men from the boys as it were. If you undersatnd it all then you wont be learning anything new. Im teasing you of course - not sure if I will make it at this point either, but we dont want them to water it down. Do we? At some point you have to do the work, you cant expect cpod to do it for you. I would suggest the backward approach. Study the vocab and pdf and expansion first, then listen. -RJ