Hello everyone. Today let's continue with our discussion of a project proposal. I'm focusing particularly on the review of the project proposal. So far we have been talking about proposing our own data mining project and specifically identifying the key components that we should include in the project proposal. By now you should have finished your project proposal. Specifically, we asked you to send me two items. One is the proposal slide and also the other one is a proposal report. What we want to do now is that we want to review each other's proposals. We'll do this through peer review. The important thing is that as you are reviewing each other's proposals, right? Try to look for things that you think, of course, help each other to improve the proposal further, but also just try to reflect upon your own proposal. This is actually very important to learning process. Let's jump in. As we said, there are two pieces of submissions. First one is the proposal slide. The general idea is that you want to just prepare a few slides. That just give a quick overview of what you're proposing to do in your data mining project. Now as a reviewer, now you are actually seeing somebody else's proposal. This is the proposal slide. You'll go through the slides. Basically just trying to understand what this proposals are trying to do. Let's look at some of the specifics. These are the instructions that were provided for the proposal slides. Now as a reviewer, you want to check. First of all, we said five to ten slides. It's not a [inaudible] it's really more like a guideline because it typically, it'll be useful for you to have a few slides. Not too few because you do need to cover the general information. But you don't want to have too many slides because too many that means that they'll take too much time, and actually you don't have a very compact way of presenting your idea yet. It's actually a very good practice. When you look at the slides, you want to say just a few slides, but then I can't gain a quick understanding of the project. That's just a general starting point. Then of course, as we have talked about, is the project title. We want to have informative title. When you're reading somebody else in the proposal, we just see the title. Do you have a reasonably good understanding of what this project is about? If you have comments, suggestions, of course, that's where you can take notes. Okay? Then we get to the specifics of what each of those the slides should cover. So that we're not seeing many slides. There just a few slides, but it should've cover some of the key components. Starting point; is the problem statement clear to you? Do you see what this classmates is trying to propose? Make sure whether it's clear to you. Then it gets to the next slide. Usually just like one slide but quickly are talking about what has been done already, and how this project relates to prior work. Again, doesn't need to be very detailed, but this is just like the slides. But it should highlight the most relevant work. Then you get to the core piece which is the proposal work. Since this is just the proposal stage and also the different projects may vary depending on how much concrete ideas that are already included in the proposal work part. That is a fine. But you want to see whether there are some good points that have been stated in terms of what what the plan is about. What are the tasks that need to be included and how they may be carried out? Also there should be one slide about the evaluation plan. At this stage, of course, now we're not talking about the concrete evaluation yet. But this is more about planning out what the person intends to evaluate. Because giving the project to your child like he or she is trying to work on given the expected outcome. Then you want to basically try to see whether the evaluation metrics, the actual comparison like are there any specifics that are related to the evaluation plan and does that make sense to you. Also, there should be a slide about roughly the timeline. Just say okay, what are the tasks that need be carried out and then in what timeline. Each of them should be finished. That's also just for you to understand how this project will progress. We also had this note about the style. This is important, I do say a lot when you're preparing your slides, you try to observe all those pieces. But many times it's much easier when you see other people's slides. Then you can see, "Oh, that's really nice. I like how the slides were organized in this way," or "Wow, I feel like those slides doesn't really look nice or good or can be better. " But that's how you really I think is very useful just to see each other's slides and pay attention to the style of the slides. Specifically, whether the overall slides are clean, whether you see the bullet's or the fonts are clear and it's easy to see. Not too many, detailed and complex information on a single slide, and of course the texts and the figures should be clear and whether there is some effective use of color. Just pay attention to others as you're reviewing their slides. The next piece is the proposal report. The slides in a way just give you a quick overview. You can probably read through the slides fairly quickly and have a reasonable understanding. But then when you go to the proposal report. That's when you can read some of the details, in terms of the more expanded discussion of the problem, of the set up, and of the proposed work, and more detailed evaluation all that. That help you to take what you have learned in the slides and then really deepen your understanding of the project when you're reading the report. Again, we have given specific instructions about the report. As a reviewer, while we're paying attention to. So sign a ways just generate the template because as a said we are asking everybody to use the ACM proceedings template. Just see whether it looks reasonable. This should be the two column format, the font size shouldn't be too small and it shouldn't be too big. Actually, there's a default settings so you should just using the template, so just make sure whether the template looks reasonable to you. Or do you see any issues? Just ways the template of the formatting. But then we get into the details I remember the specific sections. We talked about project title so you should have seen that in the proposal slides already. But then when you look at the key sections in the report so you start with the abstract. The abstract should give you a good overview or summary of what this project is about. It should be just a one paragraph or two paragraphs, in terms of texts but it should give you a good summary of what this project is planning to accomplish. Then when you look at the introduction. This is when you're reading specifically, what this problem is about, why it is important to adjust this problem. Limitations with this work and also what this project is trying to accomplish and that is what is a potential contribution. As you're reading it, just see whether you agree with the reasoning and the argument on the way. Do you see what the problem is? Do you see there's a value in addressing this problem? Do you think that the author did a good job explaining the gap or the limitations of existing work? Do you see potential contributions? This is really just front the reviewer's perspective. As those content is being presented to you. What do you feel? Is it a clear and is it actually convincing? Then you get into the next section. This is where we're talking about the related work. Related work as we said, this section just really to provide a more details of the related information. This will be prior work, this could data tools, semesters or price studies, coming with certain results. They also should actually give a good description of what has been done. But also showing how his or her work is going to build up like the prior work. All right. Then proposed work. This is the main section or it can be break into multiple sections. But in this section you should be read in a clear fashion like how the projects will be carried out. What are the core components? How things will be carried out in particular procedure or, what are some optional or export random may be changed or later versus some other core pieces that have to be accomplished before the next pieces may happen. Again, keep in mind this is just a proposal, it is not a clearly, say, everything planned out already. But there should be some good ideas and good reasoning about why you want to do this and why you're not doing it this way. Then evaluation. This is really more about the planning. Planning of the evaluation. Here, there may be some specifics about what data, what evaluation metrics, what will be used and what is mastered or may be used for comparison. Whether there'll be some planning in terms of the training set versus a testing set. Again, it's like as you're switching your position from writing the report and now you're reading somebody else's report. See if now this actually makes sense to you and also coming to reflect in terms of your own proposal and see whether such things that can be improved or the things that is really good, this is really clear and it does a good job in terms of convincing the reviewer. Discussion section, this actually has a bit of more of the pieces in terms of the timeline and also the current status and also potential challenges, potential changes all that. Again, you're really reading about the reasoning. As we said, this is the proposal stage. We don't know exactly what are going to find so we just need to plan ahead but then be aware or be prepared knowing that there may be certain things that may be adjusted later on, if needed. Again, as you're reading this section, you basically trying to review and determine whether the author has done a reasonably good job, reasoning about the whole project. Then conclusion. This is a way that sound as a summary. Because when you finish the whole project, you will still provide a good summary and also by the time you actually finish the project then there will be more. But the conclusion should still give you a good refresher of overall what this project is about. There's also one section, so this is the references. This is just particularly in terms of all the related work that you're citing or other prior study are citing. Just make sure it's cited in the proper fashion. You should have the author names, and the venue, the title of the article, and the time information if you know when it was published. Those shall just give you good information in terms of reference so other people can easily lookout. As you're looking in the references to see whether they are clear and whether there's any missing information. This is a proposal report. This of course, will take a little longer to read compared to the proposal slides. But hopefully, jointly when you look at both slides together, and by zooming into the specific sections or components that we ask each student to put an ink. Now you can sit in with the reviewers upon review and just see whether this proposal makes sense to you. That's really more like the key questions. Number 1 is just that, is the project idea clear to you? Because the number 1 thing is that if you want to say this is what I want to do, you want to make sure other people understand what you're trying to propose. Presentation, this is the first most important part is that just people understand what are you're trying to do in your project. Clear presentation. Then the second one is a value. I think that this one is important and we talk about it multiple times. But many times I think we all probably have that tendency of just to go ahead and do it. Oh, I have this interesting idea and just go and do it. But we do need to really emphasize more in terms of value. Because as I said, you want to make sure you're making potential contribution through your project. You're not just spending your time and effort on a project that you may produce some results, but the results may not be particularly helping in any way. As a reviewer, do ask for this question, do you see good potential value of the project? Of course, this is just a course project. We're not trying to make maybe a huge impact or change the world in many ways, but you want to make sure that the outcome of the project has some value that can be helpful in some downstream scenarios. Then you can ask a little bit more into specifics. If you understood what the project is about, you know the core tasks in this project. As I said, the author should provide justification of why those tasks. Now, you as a reviewer, think about, okay, I know those are the core tasks in this particular project, I understand what the author is trying to do. What do I think about those tasks if I'm to work on this project? What would it be? Do I think of those tasks would be reasonable or other things that can be better or what the other things that can be considered? As you're talking about this, so this is your reasoning and also building on top of the author's reasoning about why those particular tasks. Evaluation plan. Always ask about the evaluation plan. Are you convinced about this evaluation plan? Do you think that the author has done a good job planning out or do you see potential roadblocks in hopes of evaluating this particular project? Build on a couple of that. Just a very important question also. Next one is about the feasibility. We talked about it in the previous lecture. Well, is that there are many interesting ideas. You've been given a particular problem and there may be different directions you can explore, but you want to make sure this is a feasible project. Of course, there are many aspects that may impact the feasibility of a particular project, but as a reviewer, you want to try to synch from the different perspectives because for any project should be carried out successfully. Make sure the data is available and actually the specific types of data. Also, we should have talked about it in the proposal, what data they're trying to use. Now you're trying to figure out okay, is that reasonable? Does that actually provide the information needed for the particular types of analysis to be done? Then specifically the tasks. Some of those tasks may be too big or there's a lot of dependency and they may not be easily carried out. Or yeah, this is actually nicely planned out sequence. I've seen that those tasks do complement each other in terms of answering the big question. So all those is your evaluation or as a review of how you think about the feasibility. All right. Also, think about whether the evaluation has specific requirements which may make it much harder to actually do the evaluation, and whether the timeline is reasonable or this is really probably too tight towards the end and you may not have enough time to finish those tasks. Again, think from the reviewer's point of view, how do you think of this project? Do you think this project is feasible? Then gets to this of more like open-ended question for you as a reviewer is that, as you're learning about this particular proposal while looking at some of the specifics, do you see ways to improve further? So this is your suggestion as a reviewer to the author of the proposal. As we said, all of us have different backgrounds, different mileage, different interests, and different skill sets. That's why we can read a long fine chat and help each other in terms of just different perspectives. Don't feel that you have to be expert in a particular application domain to give good suggestions, but just based on your understanding what subject or based on your understanding for all the data mining concepts and specific techniques we have covered so far. See what you can suggest or you could say, I don't have a new suggestion, the project sounds great. That's fine. You can also provide that as a feedback. But if there are more specific aspect you think it could help the author, please do. Specifically, think about data, if you are aware of other data sets that I think this project can benefit from, please mention that in your feedback or if there are data tools that you have used or you think it could be very useful for this particular project, again, please mention that. Then tasks. If there are other tasks or slight adjustment of the proposal passed, all those sometimes you just providing your opinion. It doesn't have to be, can I say, perfect or in any way. It really just what do you think about under your suggestion. Also think about evaluation or if you're aware of other prior work because you may have to say, "I have read about this one or I have seen this particular website and may have some further information", or you found some further information. This is really again, as I said, you're helping each other. Also coming on the presentation, as a saddle we're asking on the previous slide that whether the project idea is clear to you. If there are pieces that are confusing or not clear to you or couldn't be explained further, point that out. That would help the author to improve the presentation further. Last but not the least because here, you are acting as a reviewer of others' proposals but that this is also a good learning opportunity for yourself. As you are observing other people's proposals and reading off some of the details, think about your own project. Reflect upon your own project. Say this is what I propose. Now after seeing other people's proposal, do you see ways to improve your own proposal further? Again, this is no right or wrong answer but just see whether you see something useful that you can leverage for your own project. With that, we are done with the project proposal stage. You'll have each reaching your own proposal including the slides, the proposal report. You have done good reasoning of planning out your project. Now, of course, we'll just get to the actual process of finishing project. Just a few things to keep in mind. First, I was sad to write this project proposal stage. Things are tentative. We don't have everything planned out, we don't know exactly what we're to find, and the methods may change or the validation may change so those are fine. Hopefully, you're starting with a good plan with your current proposal but keep in mind that things can change actually. As you make progress, you should actually look for ways to further improve or adjust your proposal. After that, our next milestone is the project checkpoint. This is really intended as more like a middle way status or check. We just want to make sure that yes, you started very nice live proposals. There are some very interesting ideas and passing you'd like to accomplish a little bit down the road. Now we'll come back and just see how things are doing. Are you making progress out there, are there others on track, and do you see any changes? That's more like a bit of status or check. Of course by the time we get to the end of the term that is when I'm going to finish and you're going to finish your final project report. We'll get to that as we move along in this project. Last but not the least, this is the keyword or key phrase I will keep reminding everybody throughout the project is about analytical thinking. By now I hope you have a bit of a better understanding of this whole process of just reasoning about everything starting with your proposals, starting with your project, the problem setting, the messes, the related work, evaluation, and all of that, is that any piece should have some good reasoning behind it. You know what you're doing, why you're doing this way, what does it mean if you're seeing this, why you choose this approach versus other, because that is really the fundamental value of being a data scientist. It's not about being able to apply a particular message on a given data set, it is about reasoning about your approach, reasoning about the results, and seeing why I'm doing this way, and why this message, or why this result is important. So with that, we'll finish our proposal stage. We'll check again for the checkpoint part but in the meantime, yes, continue with your project and I hope you enjoy the process. That's all for today. Thank you.
