Progress Tracker Updates

catherinem
November 29, 2010 at 08:31 AM posted in General Discussion

 

We received a great deal of feedback from you about our progress tracker, which was created earlier this year. We've taken many of your suggestions and integrated them into our simplified, updated version. Today I'd like to introduce some of these changes.

Before replacing the old progress tracker with our updated version, we'd like feedback from the community at large. We've already received some great feedback from users who tested the progress tracker for us (thank you!) And would now like to open it up to everyone. You can check out the new version here.

This is what the current progress tracker (the one the update will replace) looks like:

We've broken up progress into different sections to make it clear where this information is coming from (lessons studied, test scores, etc.) and what trends your progress is taking.

 

My Lessons

In this view, you can see the distribution of the lessons you have marked as studied across different levels. You can also see, in green, the number of lessons at each level ChinesePod recommends you complete before moving up a level. Your studied lessons (in dark blue) are a percentage of total recommended lessons (in light blue) at each level.

 

Placement Test

Your next graph is a placement test graph. If you've never taken a placement test, you can click on the link to do so. The more times you take the test, the more points on the graph you'll have. All of your tests will be plotted together, regardless of length (we have progress tests of different lengths).

 

Level Test

The next graph is similar to the placement test. This is the level test. You are tested with questions from one level only. If you've taken this test at different levels (intermediate and upper intermediate for example) then you'll have a different graph for each level. Each point on the graph is a different occasion on which you've taken the test.

 

Teacher Service users (Guided, Executive, etc.) will see this bar graph (not available to basic and premium subscribers) at the top of the Progress Tracker page, which indicates how far along in their course they are.

 

Send us your feedback!

We truly hope you find these changes helpful. Our central goals were making it clear what data was being represented and allowing for a more straightforward representation of learning trends for each individual user. Of  course, if you have questions, comments or feedback, please contact us at support@chinesepod.com, respond to this post in the comments section, or PM anyone on the team! We hope to formally make the change from the old progress tracker to this updated version next Monday, December 6th.

 

If you'd like to comment on the Progress Tracker Blog Post, you can find it here.

 

Profile picture
pretzellogic
December 11, 2010 at 06:12 AM

The old progress tracker comes up for me by default.  Is this something i'm supposed to change by myself?

Profile picture
paulinurus
December 09, 2010 at 04:36 AM

It will be great if Cpod would bring back the previous placement test. It was a simple yet effective test to determine which level a poddie is at in terms of comprehending spoken Chinese. The old test contained a series of audio sentences spoken in Chinese (gradually more and more difficult) and all we had to do was click 'yes' or 'no' whether a sentence was understood. Once you can no longer understand the Chinese, you press stop, and the system will indicate a level (newbie, ele, inter, upp inter etc) you're at.

The current two tests are based on your ability to read and write Chinese (characters or pinyin).Great for the people who are either taking formal Chinese exams or are keen in learning to read and write Chinese, They study and study writing Chinese, periodically take these two tests to get exhilarated and motivated seeing the two lines sloping up. However, these two test are of no interest or even demoralizing for those of us who are only interested in listening and speaking Chinese, which (my guess) is the majority of Cpod's subcribers who do not have a Chinese background. Since we are not interested in reading and writing Chinese,these two tests will continue to place us at the Newbie level even though we are able to understand spoken Chinese at a much higher level. In my opinion, it is also just plain wrong to say " The ChinesePod Placement Test is designed to give you an idea of which ChinesePod lesson level might be right for you." And then to guide that 'after 80 lessons of Ele you should tackle some Intermediates'.

Not the first time a completely useful feature in the past has been scrapped and replaced with something quite convoluted, complicated, and perhaps of not much used. by most. I could name a whole string of these in the past two years, but it'll just take too much of my time.

Profile picture
paulinurus
December 11, 2010 at 08:36 PM

In any language text book course, the written language is included otherwise the book is incomplete. But there are many spoken language courses via CDs where no text is included. I started learning Mandarin with "Instant Immersion Mandarin Chinese" an 8 CD spoken Chinese course with no text included at all. ABC (American born Chinese), NBC (Norweigian born Chinese) and other foreign born Chinese kids could speak Mandarin very well if their native parents use the mother tongue language at home, but these ABC or NBC themselves won't know how to read or write Chinese. So it is not necessary to know how to read and write a language in order to learn how to speak it.

As for pinyin, it is certainly helpful to learn tones, and it is essential to be able to type Chinese with a computer. However, I think taking a dictation test using pinyin is quite an inefficient way of testing for spoken Chinese. It takes more time spent to memorize the tones to the pinyin learning, and for what. just to take a dictation test in pinyin? Learning tones via ear is much more effective than learning tones via written pinyin. In fact, I've seen even native speakers go wrong when assigning tones to pinyin words. Just try it.... get a native speaker to assign tones to various chinese words in pinyin and see it they can do this correctly. I've found out that it is best to learn the actual characters if one wishes to learn how to read and write, rather than use pinyin. Pinyin is only good for romanization with tones and for using the computer to type chinese.

As for some of the responses here saying that "it is only a test, don't take it seriously, don't take the tests if you don't like, ignore being placed as a newbie... .etc" I think that is exactly what these folks will do ....as Italians'll say " fuggeddaboudit"

Profile picture
sydcarten
December 11, 2010 at 08:20 PM

I would agree with that.

My primary focus is on learning spoken Mandarin.

As an avid reader, I have placed reading Chinese characters as a strong secondary priority.

I have however ranked writing Chinese characters as a very low priority. The unlikelihood of my needing this skill here in Australia teamed with the amount of time it demands is a dealbreaker for me

Profile picture
paulinurus
December 11, 2010 at 07:43 PM

Calkins, I do write Chinese characters using Microsoft's language input (type pinyin and the program writes characters) and yes a holistic approach will certainly help to reinforce the learning of spoken Chinese. However, for most of us living outside China, spoken Chinese is naturally the main objective for learning Chinese. Since time is so limited (shared between work, family, and 在学习中文) using leverage to get the most effective use of our time is of utmost important.... it's the 20/80 rule... 20 percent of effort to get 80% progress pursuing spoken Chinese. As you know, the most time consuming aspect of learning the Chinese language is learning how to write. Reading a Chinese character is one thing, writing that character is entirely another thing. Unless you've spent a lot of time practicing and writing characters over and over again (which school kids do when they are learning how to write), you will not do well in a dictation test. And that's why poddies who have not made a conscious effort to learn how to write Chinese characters will not do well in Cpod's dictation test and will for many years deemed as newbies even though their spoken Chinese is well above the newbie level. There lies the fundamental flaw in the test, and to describe it as " The ChinesePod Placement Test is designed to give you an idea of which ChinesePod lesson level might be right for you." is like rubbing salt in a wound. And that's why I would urge Cpod to bring back the old test and make it available together with these two current tests.

Profile picture
chris.k
December 11, 2010 at 05:57 AM

Not everyone needs or wants to learn characters, but surely there aren't many non-native speakers who could understand something in spoken Chinese, but not in Pinyin? It's a fairly simple phonetic(ish) system, and I can't imagine learning the language without some exposure to it. Same with any language- your main goal might be to communicate verbally, but I'd find it pretty strange if someone did anything beyond a basic survival course in Italian without encountering the written language.

This old test you described does sound great, though, and I'd love to have it in addition to the current ones.

Profile picture
paulinurus
December 11, 2010 at 05:31 AM

When anyone wants to communicate something in a foreign language, whether it is in Spanish, French or Chinese, would he/she ask a native friend to write it down or to speak it? It is just human nature that most poddies subscribe to CPod first and foremost to learn verbal Chinese instead of written Chinese. Of course, there are exceptions like yourself (since you've challenged my assumption) who are at CPod primarily or equally interested in written Chinese, but I think these subscribers are in the minority. The enormous extra time and effort needed to learn written Chinese (and for what?) would ward off most people. That is why it is a paradox that Cpod would expend time and money to develop a "placement test" based on written Chinese. I think it is just poor customer service by taking away the previous test, if not a slight. Why the insistence that poddies learn written Chinese in order to take a placement test?

Profile picture
calkins
December 11, 2010 at 05:29 AM

Paul, I don't think it's necessarily true that learning to read and write Chinese will hold back your spoken Chinese. I think that holds some truth (perhaps in the pronunciation department), but I actually believe that writing and reading improves your spoken Chinese. Initially, maybe not so much, but eventually when you can write full sentences (even basic ones), your grammar will improve dramatically. I think the key is writing full sentences, not just individual characters. Of course, that takes time.

Also, I think writing reinforces your learning of the pronunciation and tone, so it's a little like review of the spoken part.

But definitely like you say, it's time-consuming and a lot of people just don't have the time to focus on writing/reading. If you do, my opinion is that it is invaluable to the overall improvement of your Chinese.

Profile picture
paulinurus
December 11, 2010 at 04:57 AM

Actually the previous placement test was a breeze to do ... took only about 5 to 10 minutes, and was fun, and effective. I took the test every now and then and it did indicate my progress from newbie to low intermediate. Sure, it is not scientific, but really do we need an accurate assessment , even if there is such a thing at all? The reality is we don't even need a placement test to tell us which level of lessons we should be tackling... when you tackle the lessons, you'll know.

As for learning how to write Chinese, you do it at the expense of your progress in understanding verbal Chinese. Writing characters is a whole ball game by itself, which will require hours and hours of extra study and practice. When Skritter was introduced at Chinese Pod last year, I spent a month or so learning how to write characters and quickly realized that it is going to hold back my progress in verbal Chinese. Say it'll take a year at Ele level for verbal Chinese, and two more years at Ele level if you insist also learning how to write. OK, so some may be smarter or have more free time to dedicate to learning Chinese, Which ever case, the bottom line is that he/she will stay stagnant for many more months at newbie or ele level due to wanting to learn how to read and write characters.

Profile picture
waiguoren
December 10, 2010 at 12:24 AM

Yeah, spot on. I had a friend who came to visit me in China, and he said 'nihao' and 'xiexie' to absolutely everything, and people where complementing him on how 'good' his Chinese was, but once you can string a few words together, the 'your Chinese is very good' compliment seems to go out the window...

(I haven't figured out how to use this reply thingy correctly yet, always seems to be the wrong person I'm replying to - or myself)

Profile picture
bodawei
December 09, 2010 at 11:08 AM

The nice thing about being in China is that Chinese people will say you have 'such wonderful Chinese' even if you are a Newbie. :)

Actually the more my Chinese improves, the less often I hear this remark. There is a nice little cultural lesson there somewhere.

Profile picture
ouyangjun116
December 09, 2010 at 10:29 AM

paulinurus,

How can you be so sure that most people on here do not want to learn to read and write Chinese?

Profile picture
waiguoren
December 09, 2010 at 10:20 AM

Yes, I agree. My listening comprehension is way ahead of my reading/writing ability and this is reflected in the test results.

I disagree on your viewpoint on the previous test, it may be 'demoralising' ('it only makes me more determined') but is certainly more 'scientific' than just listening to some sentences until you can no longer understand them.

Also, I started out only interested in speaking/understanding Chinese, but as time goes on, I'm becoming more and more interested in learning to write (and read) in Chinese.

Furthermore, it is only a 'test' - no need to take it too seriously, and we don't have to do it if we don't want to! (Granted: can be demoralising, when you've studied for 3 years and the results brand you a middle-of-the-road elementary learner).

Profile picture
waiguoren
December 08, 2010 at 11:45 AM

When taking the tests (dictation section) it's a bit difficult to distinguish between ‘她’ and '他‘。maybe both can be acceptable?

I've lost a mark because I got the 'ta' wrong...

Profile picture
bodawei
December 03, 2010 at 06:54 AM

One other thing - the tests will have no credibility in terms of measuring one person's performance against another. Being Internet-based, you can, if so inclined, sit there with a dictionary, a search engine, in fact all of your materials, laid out in front of you, doing the test. And there is nothing wrong with that if it helps you learn - but the results lack comparability because each person can undertake the tests using a different set of rules. As long as you always apply the same set of rules the tests may be quite reliable.

Profile picture
hongdaqi
December 10, 2010 at 03:53 AM

好久没来了,上来看看。

Profile picture
bodawei
December 08, 2010 at 10:33 AM

Hi Catherine

Further to that, I'd just add that the thing I find most valuable is the Expansion Sentences section. Not easily tested of course, but I know whether I can read and understand and discuss a sentence without resorting to a dictionary, so I think while ChinesePod maintains consistency with these sentences it gives me reference point. At one level I can do them by myself, for review and expanding my understanding of an expression or grammar construction. At a level above that I need to go through it with a tutor.

I hope ChinesePod teachers continue to give these sentences their wholehearted attention! :)

The other exercises, while fun, are less important to me - the matching of words to meaning is okay - it can help my reading skills (guessing) if the word is entirely new. It always seems 'easy' by comparison to the other exercises for any given level.

The sentence ordering is a traditional kind of review exercise, but it is as much a test of memory as a test of knowledge of the language (difficult to separate the two.)

The 听写 - I prefer to get practice live with a tutor (I prefer to write the thing longhand than use the computer input), but I am also a little scarred by traditional Chinese teaching methods there.

Profile picture
light487
December 08, 2010 at 03:18 AM

*runs and hides from the upper-intermediate lessons* :)

Profile picture
catherinem
December 08, 2010 at 03:09 AM

Good point, bodawei. And one that I think holds for all kinds of self-testing. Of course you can alter the results by using other tools (dictionaries, etc.) but generally I think these level tests help give users a general idea of what level CPOD lessons they should be studying. That being said, if I'm told by the test results that I'm at about the intermediate level, I should of course continue to study ele's and also dip into the harder stuff (upper's).

Profile picture
bodawei
December 03, 2010 at 06:38 AM

I wonder if the ChinesePod progress tracker should be treated more as a bit of fun (like most of the ChinesePod learning experience) - and I hope I do not offend those who spend a lot of money on the ChinesePod learning tools.

The tests really are best at testing how well you do ChinesePod tests -- if you see what I mean. There is no particular meaning that can be assigned to 'Elementary', 'Intermediate', UI, Advanced and Media, other than that meaning assigned by ChinesePod itself.

If I was going to rely on them I would treat them as a measure against my previous preformance, something that is implied by 'progress tracker'.

I did one test, once, for fun, to see how it works.

If we were interested in measuring performance, I think we should start with some way of testing your understanding of a lesson. It is always going to be a limited kind of test (being Internet based it cannot test your ability to converse or answer questions with a real person), but it could theoretically measure something of character recognition, reading comprehension, listening, even writing. That's where I would like to see some progress in testing. We have that already in a limited kind of way, but I would like to see that improved, and then the results recorded. If that was possible I would start using the progress tracker. (Why aren't the results of the existing tests recordable now I wonder?)

Profile picture
cinnamonfern
December 03, 2010 at 09:07 AM

I agree. I think one shouldn't take the tests too seriously. If you don't feel that it matches your level, ignore it or take it again. It's just a tool, a guide, not the law. If you don't like the tests, go back and listen to some old dialogue for review - you should be able to figure out whether you have improved or not. I can tell I've gotten better in the past three months and I don't need a test to prove that to myself. For my own part, I think the tests that I have taken have accurately assessed my level, or came pretty close to it (when I have not accidentally skipped pages of the test by accident). :)

Profile picture
mrluobo
December 01, 2010 at 08:09 AM

Hey Pretzellogic,

I spoke with our tech team here and hope the following explanation will shed some light on your situation:

1. For Teacher Services Students, The Progress Tracker will only display your current course in progress. By current course, I mean the lessons that were most recently assigned to you by your teacher. You have been assigned approximately 50 by Grace, and seem to be well on your way to completing that set of lessons. (BTW - Congrats on taking on such a large number of lessons - this is considerably more than many of our other Guided students.) Please let me know if I can contact you to discuss some of your study methods and tactics.

2. The remaining 150 or so lessons won't show up in your previously studied course, even if you clicked the "Manage Course" tab because the closing course feature wasn't around when you first started studying. That's right - you're officially older than some of the services we started offering : ) *开玩笑。。。

3。 If you look at your managed courses tab, then you should see your last course with Grace saved there. She did close that and it was saved in your dashboard. In the future, all of these lessons in your current course will also be closed when you've finished studying, and they will be in your saved courses as well for you to revisit when feeling nostalgic.

I hope that answers some of your questions and please let us know if you have any more.

XiaoLing - still not sure what you mean by deleting lessons. Hit me up and let me know the details!

Best,

Rob

Profile picture
pretzellogic
December 01, 2010 at 08:45 AM

Hi Rob,

Thanks for the reply. It helps clarify what's going on. Now I have follow-on questions/comments:

1) Grace has updated the status as you said, and given the progress tracker is now going to update for the existing lessons assigned, then you might want to make it clear on the progress bar that these are for "lessons assigned by your teacher since xxxxx date" or something.

2)Given my progress with Grace, shouldn't she be giving me a grade or something for each of the 50 lessons that I have taken? or maybe pass/fail? I suppose this can be extremely touchy with some students, but it would help to make us aware if we're going to be stuck at the intermediate level all our lives, or can move to upper intermediate because the teacher feels we're actually making progress toward that next level.

3) Much of the Progress Tracking here is around a mere numeric estimate for going to the next level (in other words, complete 50 newbie lessons regardless of how well you know the lessons or did them, then go to elementary, complete 80 ele lessons then go to intermediate....). I hope Cpod is working with Grace to actually figure out that maybe 120 intermediates is too many/not enough to actually go to upper intermediate. Or another way to say it, i'm starting to get bored with intermediate (but it's not like I do well with them if my calls with Grace are the indicator), and i'm not really looking forward to another 70 intermediate lessons (when under the covers i've completed maybe another 30-60 lessons that are not counted based on what you told me above). But it would be a fair independent assessment of my progress if the teacher feels that my performance on the call warrants me staying at intermediate. Then I would ask what metrics she wants to measure me by during our calls to warrant going to upper intermediate.

So some of this is likely confusing. I'm happy to do PMs if that helps clarify things.

Profile picture
pretzellogic
November 30, 2010 at 09:48 PM

so the "my course progress" is supposed to be on the number of courses Grace has worked with me over the past 55 weeks? shouldn't that be around 240 lessons over the past 55 weeks or so? not just 50 lessons?

Profile picture
xiao_liang
December 12, 2010 at 11:12 PM

Missed this response. Thanks for letting me know.

Profile picture
catherinem
December 08, 2010 at 03:07 AM

xiao_liang,

I"ve discussed the lesson data problems you're experiencing with our tech team and they're working on an update for the progress tracker that we plan to release early next week. Thanks for your patience.

Profile picture
xiao_liang
December 01, 2010 at 07:29 AM

Well it's not to do with teacher services rob so not within your remit I'm afraid. But the new progress tracker has reduced the number of studied elementary lesson I have from 70 to 56 and intermediate from 8 to 5. I did submit a message via contact us but it hasn't elicited a response...

Profile picture
zhenlijiang
December 01, 2010 at 06:26 AM

Hi Rob, you've replied here to pretzellogic signed in as pretzellogic.

Profile picture
mrluobo
December 01, 2010 at 01:57 AM

@Pretzellogic and Xiao Liang:

Hey guys,

Rob here - I'm the Teacher Services Manager and will be looking into this case.

As far as I know, for Teacher Services students like pretzellogic, you can only view your current course in the progress tracker. Your current "Course" corresponds to all the lessons your teacher assigned to you (usually monthly). That's not to say that all your old lessons from previous courses can't be viewed by clicking "Manage Lessons" on the bottom of your dashboard. At the end of your course, when you've studied all the assigned lessons with your teacher, she will close the course so nothing will appear in your progress tracker until the next course. Essentially it's a reference to give you easy access to how far along your current course you are.

XiaoLiang: Got your post in the blog, BTW! Glad you finally got a VPN to allow for the academic serenade. When you say "studied" lessons, are you referring to the lessons in your lessons tab, your courses tab, or both?

I got Zhang Feng here behind me today and will look into this, but thanks for the feedback and help in developing it.

Talk to you soon,

Rob

Profile picture
xiao_liang
November 30, 2010 at 11:50 PM

You might have the same problem as me. The update has deleted a whole bunch of "studied" lessons.

Profile picture
zhenlijiang
November 30, 2010 at 03:44 PM

Hey so that's what the current progress tracker looks like. I didn't know. I've not once seen the Your Progress Over Time graph on mine (I mentioned this issue when the Progress Tracker was first rolled out, not that it matters now since it's going to be replaced).

Thanks for adding the division marks on the y axis but I still think it's hard to see what level we've placed in, in the Your Placement Test Scores graph.

Profile picture
trevorb
December 09, 2010 at 07:23 PM

Sounds good, I'll let you know if they work....

Profile picture
catherinem
December 08, 2010 at 03:05 AM

The tech. team is currently working on the data issues (discrepancies) that many users are experiencing. We plan to upload the update early next week.

Profile picture
trevorb
December 01, 2010 at 09:43 PM

Mine fails on all browsers so I'm pretty sure there is something not quite right underneath. I've apparently done 314% of the recommended number of Newbie lessons !

Profile picture
zhenlijiang
December 01, 2010 at 06:24 AM

I haven't done "too many lessons"! The current version can't count my Studied lessons correctly anyway. The new version is showing OK for me. Guess it could be a browser incompatibility thing.

Profile picture
trevorb
November 30, 2010 at 10:43 PM

I think you have the same issue as me, you've done too many lessons so you just get a broken image thing instead of the Progress chart. Its still broken in the new version though....

Profile picture
chris.k
November 30, 2010 at 03:08 AM

One feature I'd like to see, ideally, is the ability to separate test results for the different scripts (Pinyin, Simplified, Traditional). I like to use the Pinyin tests and end-of-lesson exercises to test my general comprehension and the Simplified ones to test my literacy with characters. They're different skills, and it'd be nice to see my progress charted separately. 

Just an idea, of course- not sure how feasible it'd be, or whether anyone else would find this useful.

Profile picture
catherinem
December 08, 2010 at 03:05 AM

I agree this would be a great feature. Noted for future development of the progress tracker.

Profile picture
trevorb
November 29, 2010 at 09:14 PM

I have no idea if the new system is there or not as it still just simply does not work for me!  The old one did not and this one works exactly the same as the old one.

It seems I have been at ChinesePod too long and done too many lessons and the number confuses the length of the URL causing a blank box with a broken image.

I don't know what the fix for this would be, maybe I need to be able to choose a time period that would be represented or something?  I'm not too sure that I have even studied as many lessons as it is suggesting I have and would be happy for it to be restricted to the ones I have done the exercises on ( a very small amount) as this would encourage me to sit and do the exercises more ( a lengthy process that involves trying to get the house to myself as my study is in a corridor!

Profile picture
trevorb
December 01, 2010 at 09:27 PM

Fraid not it happens on any browser.

here is the image URL that would normally be the graph....

actually better not as I can't put a link in here as there are no tools when replying to an existing comment. I'll send a personal mail......

Profile picture
peterning11
December 01, 2010 at 02:20 AM

Hi trevorb, I've just looked into your progress tracker and I can see that you've completed 0/12 courses lessons you've been assigned. I can also see your course test score on July 9th and August 23rd as well as the percentage of Newbie, Elementary and Intermediate lessons you've completed. Can you send me a screen shot of the blank box with a broken image to me at peter.ning@chinesepod.com? Maybe it is a compatibility issue with your internet browser.

Profile picture
xiao_liang
November 29, 2010 at 09:51 AM

Heh. I kind of feel like all the feedback I sent in was ignored. Sob! 

I did report it during the beta, but similar to the error above, my total "bar" for elementary is bigger than my bar for intermediate, which is wrong, given it's supposed to be 80 compared to 120.

My comments about making such a big deal about the placement and level tests still stands. for reference, they were:

The level and placement test graphs are massive. Is there a need to separate them into separate graphs? They are given a LOT of space here. 

To be honest, the problem here is that the level and placement tests are not very useful full stop. The problem being that they draw from the entire corpus of Chinesepod lessons. I'm just starting Intermediate, but there's no way I know every character shown in elementary and newbie lessons, so my placement and level tests always undermark me severely. In fact, to test it, I just took a placement test - it's put me at 27% - newbie. When I took the same test back in March 2010, I got 24% - newbie. 
Obviously you're not looking at the tests with this exercise, but giving them such a massive space on the progress tracker puts a lot more emphasis on them, and it's actually very dispiriting (for me!) to see I have apparently made so little progress in the last 8 months, whereas realistically I know I'm a million times better than I was back then! 
A simple fix would be to have the level tests at least draw from the material in your own "studied" lessons only. Thereby giving a much more accurate reading and justify the huge amount of space given to these tests on this new page. 

Profile picture
toianw
December 03, 2010 at 05:46 AM

Interesting. I did the test a few months ago, just for fun (It wasn't actually - I get bored quite easily) and it placed me about where I'd expect. There seems to be a large discrepancy in results between different users. I suspect this is because it's just testing a fairly small subset of language skills. It's not really testing your ability to communicate in Chinese (which at the end of the day is your goal). I think if you're a regular user, you know your level (within Chinesepod) more accurately than any test of this format is going to tell you. My advice - don't let a computer tell you what level you Chinese is at. You already know what level of study is most suitable for you here - that's all you need to know!

Profile picture
light487
December 03, 2010 at 04:47 AM

I agree with what you are saying here.. The progress tracker should be aligned with the ChinesePOD measurement of level, not necessarily real-world level. If you wanted real-world level, you'd go and do the HSK or something else that has been specifically designed to test real-world level.

So when the progress thing comes back and says you are a "Newbie" but you are actually studying, quite easily and without any difficulty at all, the Intermediate level lessons on ChinesePOD then there is something wrong with the progress tracker's ability to correctly and accurately track the true level of the user in relation to the ChinesePOD measurement of levels.

There are two conflicting measurements here.. like comparing apple and oranges. There is the ChinesePOD lesson level measurement and there is the real-world level measurement.. they can't be tested by a test that is designed for one or the other unit of measurements, just like any tool that is designed for one thing can't be applied to another in an accurate way.

Profile picture
xiao_liang
December 02, 2010 at 06:24 PM

Yes, I entirely agree. Having just come back from China, I'm very aware of my failings and shortcomings in terms of language. Sadly I don't have the time for the intensive study you do, but I do my best with the tools and time I have.

I'm not arguing with anything either of you have said - I don't consider myself an intermediate chinese speaker by any normal definition, but I consider myself to be bridging an intermediate chinesepod learner because that's where I'm doing the majority of my learning right now, at that level. My question is that if the test is telling me I'm a newbie, which I do feel reasonably strongly I am not I'd suggest the test is not a suitable tool, that's all. I hope that doesn't come across as arrogant as maybe your last paragraph suggests!

I don't know. Since I guess my message isn't getting through either because I'm not expressing myself correctly, or I'm just plain wrong, I'll shut up about it and go away. It's not really important - the lessons remain good and I'll continue learning from them - I know I'm improving consistently and I'm happy with that, as ZLJ said. I just thought it was a point worth making, but I guess it might not be!

Profile picture
ouyangjun116
December 02, 2010 at 01:21 PM

I could not agree more with what Calkins just said, especially in regards to using multiple study methods. I love ChinesePod and think it is essential in my studies. I listen to podcasts daily, but in addition to ChinesePod I have a pretty solid program down.

On a daily basis I listen to a podcast or two; study from my 博雅汉语 book series; learn new and review words/characters through Anki; attend a one-on-one private tutor once a week; and work in an office where I try to use Chinese as much as I can throughout the day.

At the end of the day, in one-on-one conversations with people I understand almost everything, but I really get a kick in my "Chinese ass" (as Calkins says) when I join a fast paced meeting with many people being held in Chinese, or when I go visit the 上海海关办公室... unfortunately I need to bring someone from my team with me as I know my Chinese level is not up to par. If I were Upper Intermediate, I would think I could handle these meetings on my own with no issue... but although I study CPod's Upper Intermediate's, i know in "real world" Chinese I'm not there.

Similar to what Calkins said, continue to stay at it, be humble, know you're going to get knocked down - but always get back up and strive for that goal you want to reach.

Cheers.

Profile picture
calkins
December 02, 2010 at 12:53 PM

xiao_liang, my guess is that you are perhaps beyond "newbie," but I definitely think that Cpod's levels are not accurate (at least based on learning Chinese in a college setting). I also think that Cpod's newbie level is a bit too simplified, should entail more, and should last longer.

I'm basing this solely on my own experience, so I could be completely wrong here.

Before I moved to Taiwan in 2008, I had been studying very intensely on Cpod for a little over a year. I also had the occasional weekly tutor for one hour, but I don't think that helped at all.

Anyway, in that year or so I had studied all the newbie lessons, almost all of the elementary, and had just begun on intermediate. At the time, I was thinking I was upper elementary / lower intermediate. Six months after moving to Taiwan and taking the placement test at 師大 (a pretty decent college in Taipei for learning Chinese), I realized how wrong I was. I thought I would breeze through the listening and written tests. Ha, I understood maybe 10% of what the tester was saying. My listening skills were so poor that they didn't even allow me to take the written test!!! Man was that a kick in my "Chinese" ass! They placed me in Book 1, Chapter 1. WTH I thought. But I couldn't be happier that they placed me at the very beginning.

I've gone through 3 books there so far, and now I can see why I was more of a beginner (newbie) student than an upper elementary one. I would say now that I am elementary, perhaps upper ele. These labels are elusive, but I definitely think Cpod's levels are not very representative of "real world" Chinese, whatever that means.

I love Cpod and never want to bash it (aside from the occasional IT blast, ha ha ;-), but I think it does have some big holes. My personal opinion is that Cpod should not be the only means for learning Chinese. Those holes need to be filled somewhere else (tutor, college, etc.).

Chinese always kicks me in my arse. It always knocks me down now and then. I've learned to respect it and learned not to fool myself by thinking my level is higher than it is. The majority of us here love learning Chinese (for whatever reason)....the best thing you can do is keep at it no matter what, and no matter how long it takes, you will improve and your "level" will rise.

Sorry for the lengthy post, just thought I'd share my experience.

Profile picture
xiao_liang
December 02, 2010 at 10:48 AM

The "Level Test" tests mastery of the material across a certain level. It isn't intended to place you in a level (that's what the Placement Test is for).

So given that neither of them seem to perform either function correctly at the moment, I think it's fair to say they need looking at. Unless I really am a newbie, in which case chinesepod lessons themselves are set incorrectly, which I very much doubt.

The progress tracker itself is fine.

Still no response to the bug email I sent, by the way.

Profile picture
John
December 02, 2010 at 06:06 AM

Xiao Liang,

The "Level Test" tests mastery of the material across a certain level. It isn't intended to place you in a level (that's what the Placement Test is for).

It definitely makes sense to test you on material you've never seen, provided that a lot of it is composed of familiar vocabulary. You're right; if it's totally foreign, there is no point. But if only some of the material is unfamiliar, you're forced to use certain comprehension strategies to compensate, and your ability to do that is also a part of your current level.

That said, the progress tracker is still in beta and we realize there is room for improvement. It is certainly not our intent to discourage you! So we're taking your feedback very seriously, and we do realize that the progress tracker is currently working better for some people than others.

Profile picture
pretzellogic
December 01, 2010 at 06:33 PM

I will say I buy into this idea around testing for everything, not just the content. When I talk with a taxi driver, the conversation is manageable. But when I talk with anyone else, I end up saying that I understand about 20% of what they say. Part of that is due to the fact that they (the teacher, the waiter, the goods seller, etc) speaks really fast, I know I don't understand all the words they are saying, and some words I haven't learned yet, the speaker will have a bad accent, he/she said something I didn't anticipate, etc. I do get the impression that i'm an elementary, even though I study cpod intermediate level lessons.

Profile picture
xiao_liang
December 01, 2010 at 05:33 PM

Well, guess I must really be a newbie, and chanelle must be an elementary student then?

Profile picture
ouyangjun116
December 01, 2010 at 04:19 PM

I just took a placement test for the first just now and I'd have to say, it came out about where I'd expect it to and placed me at the middle of Intermediate. Detailed as follows:

35/50 correct on multiple choice - scoring a 100% on the newbie, elementary, and intermediate sections and getting a 40% on Upper Int and 10% on advanced.

10/25 correct on the listening... ouch (i blame it on poor speakers on my pc ;) - Got a 100% on both the newbie and elly, but only 1 on intermediate and 0 on upper int and advanced.

I do a lot of character studying outside of CPod in order to make sure I'm up to par on HSK characters and vocab. I'm glad to see that CPod's format of learning material is fitting in nicely with the HSK levels (at least in regards to common character usage)

Profile picture
xiao_liang
December 01, 2010 at 03:09 PM

_Should_ work in the same way. But we've had examples in this thread where it's consistently placed people way below their level, which suggests to me it's not functioning correctly.

Additionally, the level tests follow exactly the same methodology as the placement tests, so what's the point of them? Surely replacing the level tests with a review test would just be sensible.

Profile picture
calkins
December 01, 2010 at 02:52 PM

I also agree with John and ouyangjun116. If you are accepted to a Chinese college (to learn Chinese that is), they will give you a placement test (both spoken and written). Depending on your level, you will most likely not have heard/seen much (or even most) of the material. This way they can best place you in the most appropriate class for your level. I think the idea is that Cpod's placement tests work in the same way.

Profile picture
ouyangjun116
December 01, 2010 at 01:41 PM

Good comments, but I completely agree with John on this. If you're only being tested on material you have studied, this should be called review. That kind of test would measure your ability to retain the informaiton you have studied. That type of test would not let you know what level of information you truly know. If I studied very hard on 1 advanced lesson, then my placement test would be on that 1 lesson material and I'd be considered advanced to the placement test. Although good for motivation, just a lie to myself.

For placment tests I think of them as upper and lower control limits (for any engineering or statistics nerds out there) and testing yourself on that level of possible knowledge within those ranges whether you have seen the material or not... the score will then tell you how well you know that range of possible knowledge.

Profile picture
chanelle77
December 01, 2010 at 04:14 AM

Had similar issue, although very comfortable with upp int / adv /media my level was elly :-)

Profile picture
paulinurus
December 01, 2010 at 04:10 AM

This progress tracker is not useful for me too. When I tested myself, I'm still a Newbie, even though I can now understand about 80% of Jenny's comments in the Intermediate lessons. I'm also skeptical of a theory (is this a tested and proven academic theory or Cpod's own theory?) of testing on materials you have not studied and don't know. It doesn't make practical sense to me that a language placement testing would be based on vocab on lessons not yet studied.... maybe such a technical approack is OK if one is taking a formal exam such as HSK and is expected to accurately know tones and pinyin spellings as well as write Chinese characters, but not practical and is certainly discouraging for adults who just want to learn a foreign language. The design of the placement test seems also contradictory to your guideline that after studying 80 or so Ellie lessons, one can then tackle the Intermediate lessons.

Profile picture
zhenlijiang
November 30, 2010 at 03:58 PM

Hey XiaoLiang, you have your girlfriend and your Chinese friends and new family in China who can give feedback on your progress (some more candid than others perhaps, but all of them must be encouraging, right?). I don't have people like that. But I still don't, to be honest, turn to the Progress Tracker to see how my Chinese is/isn't improving. I have a good enough idea--from how well I do or how hard I find it whenever I try to make myself understood in speaking or writing, how well I can understand when listening or reading. Also from how hard/easy I find lessons here. I too also find issues with the tests here (it's too easy to submit a still-incomplete test by mistake, for one. my main beef--the dictation questions should have higher value, in other words you should be able to make one tiny error in the sentence and still earn a point or two for that question). But if you realistically know you're a million times better than you were back then, isn't that all you need to know?

I guess I seem to be saying the Progress Tracker here isn't too useful. Well for those people who do find it useful, it is. For me, I guess I wouldn't miss it, if it were gone one day, as long as the My Archive box stayed.

Profile picture
bodawei
November 30, 2010 at 03:41 PM

Hi xiao_liang

You raise an interesting point and I can give you some feedback from two 'placement' tests I've done outside of ChinesePod. One in Sydney, before commencing a course. There was a short introductory conversation with a teacher. The guy then handed me a book (guessing my level from the initial conversation must have influenced the choice of book?) I'd never seen before and asked me to read (out loud) for about five minutes, and then he asked me questions about what I'd read, a test of comprehension. I had never seen this book before in my life. But everyone's experience is different so giving this kind of test does help teachers to 'place' you relative to other students.

Then in China, similar thing - an HSK format test - but I had done no 'studying' for the material - everyone (from several different countries) was in the same position, being unfamiliar with the material.

I don't know if this helps ..

Profile picture
xiao_liang
November 30, 2010 at 08:46 AM

A little bit. But they're not really level tests then, don't you think? Let's suppose chinesepod is the only place I learn chinese (which it practically is). How does it make sense to test me on something I've never seen? I'm just not going to know this vocab. Otherwise how would it make sense that I've studied 70 elementary lessons, I'm listening to intermediate lessons and understanding them, yet your level and placement tests both put me as a newbie?

I just look at that and feel terrible about my progress. It's counter-productive, and doesn't help me learn anything.

Profile picture
John
November 30, 2010 at 01:52 AM

Xiao Liang,

I understand what you're saying about the Level/Placement tests being over-represented, but allow me to explain briefly why it makes sense to test you on material you don't know.

If you're only tested on material you've studied, that's just review. But if you're tested on material at your level which you haven't studied, you're not actually being exposed to 100% foreign material. Because we always focus on high-frequency language, the more lessons you study, the more the proportion of words that you already know in every unstudied lesson keeps going up.

So if you've only studied 2 Elementary lessons, the rest are going to be hard. If you've studied 80% of the Elementary lessons, however, then the ones you haven't studied yet are likely to be mostly easy for you.

In practice, this means that the longer a learner studies a particular level, the easier each subsequent lesson is, on average. The tests reflect that. I hope this makes sense!

Profile picture
xiao_liang
November 29, 2010 at 09:58 AM

Wtf, also, something has happened to the "studied" lessons! At the weekend I'd studied 70 elementary lessons. Now on the manage lessons page, and on the new progress tracker, it's only showing 56 lessons studied! Where are my other studied lessons?! The old progress tracker is showing the correct number:

newbie: 20

elementary: 70

intermediate: 8

The new one (and manage lessons page) is showing:

newbie: 20

elementary: 56

intermediate: 5

Help! Please fix this! :-(

Profile picture
coco71
November 29, 2010 at 09:26 AM

I think the new progress tracker is a great improvement. Just one small glitch: the chart displaying the percentage of my completed elementary lessons is wrong.  It correctly shows the number of lessons completed (60/80) and it event says 75% completion but the bar shows something like 45% completion.

Profile picture
catherinem
November 29, 2010 at 09:37 AM

Interesting. I'll have the team look into this error. Thanks for letting us know!