EVALUATIONOFLEARNINGINCOMPUTERBASEDEDUCATION
USINGLOGSYSTEMS
MarttiRahkilaandMattiKarjalainen
HelsinkiUniversityofTechnology
LaboratoryofAcousticsandAudioSignalProcessing
P.O.Box3000
FIN-02015HUT,Finland
Martti.Rahkila@hut.fi,Matti.Karjalainen@hut.fi
http://www.acoustics.hut.fi/
ABSTRACT
Evaluationisdifficultbutimportantpartoflearningtobothteachersandstudents.ThispaperdiscussestheideaofusinglogsystemstocollectinformationinComputerBasedEdu-cation(CBE).Theloginformationcanbeanalyzedandusedtogetheroreveninsteadoftraditionalevaluationmeasures.Furthermore,intermsofself-studyCBEmaterial,loggingcanbebeneficiallyappliedtointeractivelycontrollearning.Asaspecialcaseofinterest,methodsforusingWWWlogsysteminlearningevaluationaswellasadvancedstructuresforinteractiveWWW-basededucationarediscussed.
1.INTRODUCTION
Theevaluationoflearningiswithoutadoubtofinteresttoeveryteacher.However,itisverydifficulttodevelopanyob-jectiveortechnicalmeasurestoit.Afterall,learningevalu-ationcontainsalsophilosophicalaspectssuchaswhatlearn-inginherentlyis.Still,teachershavethroughoutyearsdevel-opedandusedsomecriteriatoestimatetheirworkresults.Thetraditionalmethodsincludeexaminationsandfeedbackforms.However,ComputerBasedEducation(CBE)allowsanotherpossibility,logging,thatis,gatheringinformationandanalyzinguser’sactions,tobeusedtogetherwithoreveninsteadofthetraditionalmethods.
Thispaperdiscussestheprosandconsofthetraditionalmethodsandpresentstheconceptofloggingfromeduca-tionalpointofview.Threemajorusesforloggingarepre-sented:1)usinglogs(files)asateacher’sreferencetoeval-uatelearningprocess,2)usinglogging(memory)tointer-activelyguideusersinmultimedia,self-studyCBEapplica-tions,and3)usinglogs(filesandmemory)togiveusersapossibilitytoexaminetheirownprogressandautomaticallygeneratereportsinmultimedia-based,self-studyCBE.All
threecasesareillustratedwithexamplesfromCBEapplica-tions.
Asaspecialcaseofinterest,WWWanditslogsystemarediscussed.AllcurrentWWWserversincludealoggingmechanismandthisinformationcouldbeusedforevaluat-ingWWW-basededucation.AstandardWWWserverlogsystemisthoroughlydiscussedandevaluatedwhatcanandwhatcan’tbeanalyzedfromthatdata.MethodssuchasuserauthenticationorHTTPcookietechniquesandtheirusefortheidentificationofusersareexamined.Furthermore,itisshownhowloggingcanbeutilizedforinteractiveusercon-trolinanadvancedWWW-basededucationsystem.
2.EVALUATIONOFLEARNING
Theevaluationoflearningisadifficultbutextremelyimpor-tantpartofdevelopingcoursesandteachingmethods.Themotivatingeffectisobvious.Butequallyorevenmoreim-portantthanitistoateacher,itistoastudenttoevaluateandmeasuretheirprogress.Onlybyfeedbackstudentsareabletounderstandthequantityorqualityoftheirlearning.How-ever,thequestionoflearningevaluationisfarfrombeingsimple:Howtomeasureifastudenthaslearnedsomethingandifso,whatwoulditbe?Attheveryend,thequestionisnotoffindingasuitablecriteriabutaphilosophicalone:Whatislearning?
Awidevarietyofmethodsforevaluation,assessment,comparisonetc.oflearninghasbeenpresentedinthelit-erature,forbothcourseandcurriculumlevel.Itisnotwithinthescopeofthispapertocovertheseall,buttwocommoncourse-levelmethodsarediscussed.
Perhapsthemosttraditionalmethodofevaluatinglearningatcourselevelisanexamination.Inthismethod,theteachercreatesatestandstudentstakeit.Afterthat,theteachereval-uatesanswersascorrect,incorrectorsomethinginbetween
0-7803-5643-8/99/$10.00c1999IEEENovember10-13,1999SanJuan,PuertoRico
29thASEE/IEEEFrontiersinEducationConference
12a3-16
Session12a3
andfinallydescribestheresultsasnumbers.Thestudentcanthencomparehis/herresultstoanabsolutescaleortoresultsofothersandthusevaluatetheirperformance.Theteacher,ontheotherhand,takestheaverageresultsandusesthemforanestimateoftheoveralllearningofthestudentstak-ingtheexamination.Butcanwereallysaythattheseresultsdescribelearning?Ifthetestwascleverlydesigned,proba-blyyes,atleasttosomeextent,butingeneralthismethodisfartoodependentofthetestitself,thecourse,thegoals,thematerialetc.Oftenthesituationisthattheteacherhastoadjusttheoverallscaleleavingnothingbutahunchoftrueevaluationoflearning.Still,thereisnoneedtoabandonex-aminationswithoutcarefullyconsideratingitfirst.Despitetheirproblems,examinationsareveryoftentheonlypracti-calalternative.
Anothercommonlyusedmethodwithcoursesisafeed-backform.Studentsaregivenaformandtheyareexpectedtouseittoevaluatetheirownlearningand/orcoursewithsomekindofpredefinedscaleorbyshortwrittenanswers.Itiscustomarythatwhenaformisusedtogetherwithanexam-ination,theydonotaffectcoursegradingandusuallythereisapossibilitytofulfillandreturntheformanonymously.Unfortunately,oftenthefeedbackformshavebeendesignedonlyfromtheviewpointofateacherandthebenefitoffulfill-ingoneremainsunclearforastudent.Thereareexceptions,ofcourse,andawelldesignedfeedbackformcangivebothstudentsandteachersequallyusefulinformationforlearningevaluation.
Therearealsomoreadvancedmethodssuchascourseas-signments,laboratoryexperimentsetc.thatgiveastudentapossibilitytoapplythethingsthattheyhavelearned.Thecourseassignmentscertainlygiveamoreaccurateestimateofthelearning,butforteachers,itissometimesimpossibletoorganizesuchinpractice.Combinationsofthemethodsdescribedcanalsobebeneficial,seeforexample[2].
3.THECONCEPTOFLOGGING
Incomputerscience,loggingisnormallyconsideredassav-inguserordebuginformationtoafile.Itisaverymuchstan-dardtechniqueto1)calculatetheusageofthesoftwareand2)tracebackpossibleproblemsituations.Inclient-serverapplications,loggingisaverysolidandnaturalpartofthesystem,becausemuchoftheloginformationhastobeex-changedbetweentheclientandtheserveranyway.There-fore,forexampleallcommonInternetserviceslikeWWW,ftp,emailetc.includealogsystem.
However,inthispaper,theconceptofloggingshouldbeunderstoodinamoregeneralway:asexchangingandstor-inginformation.Theinformationcanthenbesavedintoafileoronlykeptinamemorywhileanapplicationisused,dependingonwhatitisusedfor.Whatinformationisinthis
senseloggedalsodependsonaparticularapplicationeventhoughcommonparametersdefinitelyexist.Thusthecon-ceptofloggingshouldactuallybethoughtofasincludinguserrelatedmetadatawiththeoriginalinformation.
Itshouldbenotedthattheconceptofloggingmayincluderestrictionsbylaw.Insomecountriesandincertaintypeofservicesorapplications,loggingisregardedobligatorybylawsandinsomecountries,eventhesameones,thelawsprohibitanyloggingthatmayexposeorrelatetopersons’trueidentity.
Therefore,wheneveralogsystemthatauthenticatesusersisused,theauthenticationmechanismshouldnotrelatetoperson’strueidentitybutprimarilyidentifyanddistinguishtheusersfromeachotherandsecondarilyidentifytheyasindividualpersons.Inthefieldofeducationthisinfactcouldbeaproblem,thinkofforexampleelectronicexaminations.Still,quiteoftenitisonlydesiredtodistinguishthestudentsasindividualstudentsratherthantrue,individualpersonsinlegalsence.
4.LOGSASTEACHER’SREFERENCEIn1994-1995atutoriallevelCBEapplication,“Introduc-tiontoSignalProcessing”[1]wasdevelopedattheHelsinkiUniversityofTechnology,LaboratoryofAcousticsandAu-dioSignalProcessing.Theapplicationisamultimediaself-studypackage,butithasbeenusedasapartofthecourse“FundamentalsofAcousticsI”whichistheveryfirstacousticscourseforstudents.“IntroductiontoSignalProcessing”isanAppleMacintoshapplicationandstudentsusedtheprograminadedicatedworkstationlocatedinthelaboratory.
Theapplicationconsistsof“pages”(windows)withei-therstaticinformation,interactivedemonstrationsorques-tions,and“chapters”,collectionsofpagesthatcoveracer-taintopic.Thestudentswereallowedtogothroughthesetopicsinanyordertheywished,buttherewasalsoade-faultpaththatcouldbefollowed.FromtheverybeginningofthedevelopmentprojectitwasclearthatinformationwasneededtoevaluatethesuccessandusageofthisCBEappli-cation.Fromteacher’spointofview,thequestionsofinterestincluded:
Howmuchtimewouldthestudentsspendwithit?Howmuchmaterialwouldtheygothrough?Inwhichorderwouldthestudentsstudysubtopics?Aretherepages/topicsthatstudentswouldspendalotmoretimeonthanothers?
Aretherepages/topicsthatstudentswouldskip?Whatistheaveragetimestudentsspendontheapplica-tionorpage?
November10-13,1999SanJuan,PuertoRico0-7803-5643-8/99/$10.00c1999IEEE
29thASEE/IEEEFrontiersinEducationConference
12a3-17
Session12a3
Wouldthestudentstakethedefaultpathorwouldtheyusethepossibilitytostudythetopicsinsomeotheror-der?
5.LOGSASINTERACTIVEGUIDES
The“IntroductiontoSignalProcessing”containedalsoan-otherloggingmechanism:itkepttheinformationofvisitedpagesinmemory.Becausethepagesweredividedintotopic-specificgroups(chapters),itwaspossibletokeeptrackofthechaptersandautomaticallysuggestuserswithtopicsthattheyhadnotcoveredyet.Furthermore,thevisitedpageswerealsoattachedtonavigationhistorythusallowingstu-dentstorefertoanypagetheyhadalreadyvisited.
Inself-study,multimediaCBE,suchbehaviorisveryuse-fulbecauseitallowstheapplicationitselftosuggestthestu-dentwhattopicshe/sheshouldstudynextand,furthermore,relationshipsbetweentopicstobeincontroloftheCBEde-veloper,thatis,theteacher.
“IntroductiontoSignalProcessing”utilizedthisbehaviorbycontaininganinteractiveexperimentthatstudentswereallowedtoexploreonlyafteralltopicshadbeencovered.Theexperimentitselfcombinedinformationfromtheothertopics.Ingeneral,thiskindofcontrolissomewhatessentialinself-studyCBEtomakesurestudentshavethenecessarybackgroundinformationbeforemovingontomoreadvancedmaterial.Itisalsousefulincaseofelectronicexaminations.
6.LOGSASSTUDENT’SREFERENCE
IfaCBE-applicationcontainslotsofmaterialorthematerialconsistsofdifferenttopicsrelatedtoeachother,itisneces-sarytogivestudentapossibilitytoestimatehowmuchtheyhavedonealreadyandhowmuchtheystillhavetodoinor-dertocoverallorcertainamountofthematerial.Oneusefultoolforthatisthepreviouslymentionednavigationhistory,but“IntroductiontoSignalProcessing”containedalsoan-othertoolforthis:thestatuswindow.
Afterloggingin,thestudentcould,atanystage,choose“Status”fromamenu.Thestatuswindowthenpresentedin-formationsuchasthetopicsalreadycovered,thetopicstodoandtotaltimespentsofar.Itcouldhavealsoincludedcomplexestimateslikeatimeestimatehowlongitwouldtakeforastudenttostudyeverything(basedontheaverage“speed”)etc.butsuchwerenotimplementedin“Introduc-tiontoSignalProcessing”.
Fromthepointofviewofself-evaluationoflearning,thestatustoolisveryusefulandatitsbestquiteaccurate,de-pendingofcourseontheparticularCBE-application.
Anotherinterestingideaistouseloggingforautomati-callygeneratingreports.Anexampleofthistypeofusewasincludedina“Psychoacoustics”CBE-package[1].Theap-plicationcontainedtheoryanddemonstrationsofpsychoa-cousticalphenomenaintheformoflisteningtests.Thestu-dentscouldforinstancemeasuretheirownfrequencymask-ingcurvesbyperformingaseriesoflisteningtests.Theau-
Inordertogetanswerstothesequestions,alogsystemwasimplementedintotheapplication.Thefollowinginfor-mationwaslogged:
login:coursename,username,studentnumber,date,timepageopen:symbolnameofthepage,universaltimepageclose:symbolnameofthepage,universaltimelogout:time
Theinformationwassavedastextandlaterimportedtoaspreadsheetprogramfordetailedanalysis.Afterusingtheapplication,studentswerealsoencouragedtofillinafeed-backformthatcontainedquestionsrelatedtoboththecon-tentsandtheimplementation.
Theuseof“IntroductiontoSignalProcessing”wasessen-tiallyonlyasmallpartofaregularcoursethatincludedanexaminationastheprimarycourseevaluationmethod.Theapplicationwasnotintendedtoreplaceanyothercourseorcoursematerialbuttoprovideadditionalmaterialtothereg-ularacousticscourse.TheCBE-materialwasnotincludedinthecourseexaminationandthatiswhytheevaluationhadtobebasedonsomeothermethod.Therefore,inordertoestimatelearningwiththeCBE-application,thelogswereanalyzedandtwocriteriachosenaslearningmeasures:theoveralltimeastudentspentwiththeprogramandtheamountofpagesstudentbrowsed.Thesecriteriaweregivenmini-mumvaluesbasedonthecontents.Studentswhospentmoretimeandwentthroughmorepagesthantheminimumvalues,weregivenadditionalpointsintheexamination(5%ofthemaximum).Feedbackforms(paper)wereusedasabackupmethodtoensurelogevaluationresultsespeciallyinpossibleproblemsituations.
TheCBE-applicationbeingaseparate,self-studyexper-imentwithoutanyrelationtopreviousorparallelmaterialpreventsathorough,comparativeanalysistobeincluded.However,theoverallresultsfromthefirstgroupofstudents(approx.80)areasfollows:AcomparisonoftheresultsofthefeedbackformanalysisandtheloganalysisclearlyshowedthatstudentswhoreallyconcentratedandspenttimewiththeCBE-application,actuallylearnedsomethingfromitandthatthechosencriteriawouldbesuitableforlearn-ingevaluation.Additionalloganalysisresultslikeaveragetimespentonapageshowedthatstudentsspentmoretimeonpagesthatcontainedinteractivematerialbutotherwisenomajordifferencesbetweenthepagescouldbefound.How-ever,thiskindofresultwasratherexpectedintheoriginaldesigngoalsforthisCBE-application.
0-7803-5643-8/99/$10.00c1999IEEENovember10-13,1999SanJuan,PuertoRico
29thASEE/IEEEFrontiersinEducationConference
12a3-18
Session12a3
tomaticallygeneratedreportinthiscasewasstudent’sownfrequencymaskingcurve,whichhe/shecouldeasilycom-parewiththeoreticalresults.
TheautomaticreportgeneratingisausefulconceptineveryCBEapplicationthatinvolvesmeasurements(realorsimulated)ortest/question-likeinteractivity.
Thestudentscanalsobegivenaccesstotheirownlogfilesafterwards.Analyzingthelogsbythemselveswouldeasilypointoutissuesandtopicsthatwerenotthoroughlyexaminedandestimatewhattopicsshouldbegivenfurthernotice.Basically,theanalysiscouldbesimilartotheoneteachersperform,butinthiscase,itwouldonlyevaluatethelearningprocessofanindividualstudent.
7.WWWLOGSYSTEM
Inthelastfewyears,theWWW(WorldWideWeb)hasbe-comeperhapsthemostimportanttoolforCBE.Fromtheverybeginning,WWWservers(httpd)haveincludedalog-gingmechanismaswell.LikemanyotherInternetservices,thehttpdlogsystemhasbeendesignedmainlyfor“hit-rate”typestatisticalanalysisordebuggingpurposes,butinfactitisoneofthemostflexibleandwell-designedlogsystemsaround.
Tobeprecise,theWWWlogsystemisasumofthreecomponents:1)theserversoftware,2)theclientsoftwareand3)theHTTPprotocol.Theinformationthatiswrittentothelogdependsontheserversoftwareandalsotheparame-tersthataresentbythebrowser.TheinformationexchangeisdonewithinthecapabilitiesoftheHTTPprotocol[4].Atypicalhttpdlogcontains(exampletakenfromthedocumentationofoneofthemostpopularWWW-servers,apache(http://www.apache.org/)):hostidentauthuserdaterequeststatusbyteswherethetokensare
Ifatokendoesnothaveavaluethenitisrepresentedbyahyphen(-).
TheaboveiscalledCommonLogFormat(CLF)andtheinformationiswrittentosocalledTransferLog.Apacheserveralsohasapossibilitytousecustomizedlogformatswithanumberofadditionalkeywords.However,theCLFassuchisadequateformostpurposesexceptfortwoparticulartokens,RefererandUser-Agent.TheRefereristheurltothepagefromwheretheusercamefrom(viaHTML-link)andUser-Agent,anid-stringofthebrowsersoftwaresend-ingtherequest.ApacheservercanbeconfiguredtostorethesedirectlytoTransferLogwithcustomlogcommands“%Refereri”and“%User-Agenti”.ErrorsareusuallywrittentoaseparateErrorLogwithsimilarinformation+possiblysomedetailsoftheerroritself.
ThelogsystemsinotherWWW-serversvarybothinformatandincustomisabilitybutthedifferencesarequitesmall.
Onabusyserverthelogfileseasilybecomeverybigandthusdifficulttohandle.Hencethereisanumberofspecial-izedWWWloganalysistoolsavailable,forinstanceanalog(http://www.statslab.cam.ac.uk/˜sret1/analog/)orWUsage(http://www.boutell.com/wusage/).Themostadvancedonesareveryfastandcustomizableintermsofanalysisparametersandinputoroutputformat,butusuallytheyprovideonlybasicstatisticalmeasuressuchashitratesetc.
8.LEARNINGEVALUATIONWITHWWWLOG
SYSTEMFromthepointofviewoflearningevaluation,itisimpor-tanttonoticethatthebasicloggingmakesitsomewhatim-possibletotracebackindividualusers’requests.Astan-dardsolutionistouseauthenticationtorestrictaccesstothecontents.MostWWWserversandbrowserssupportauthenticationandinitssimplestform,userauthenticationiseasytoimplement.Usuallyauthenticationinvolvesalsoaccountmanagement,preferablyasautomaticaspossible,whichisnotsimpletoimplement(notbeingverydifficulteither,though).Agoodmanagementsystemwouldallowstudents(oranyone)toregisteron-lineandcreatetheneces-saryaccountimmediatelyaswellasmethodsfortakingcareofchangingpasswordorremovingaccount.Ifuserauthen-ticationisused,itispossibletodooff-lineevaluationandanalysissimilartoonepresentedinsection4.
AnotherdisadvantageofthebasicWWWlogsystemisthattheinformationiswrittenonlyonarequestbasis.Toputitinotherwords:itdoesn’trememberanything.Inor-dertouseloggingforpurposessuchasinteractiveguidanceoron-lineevaluation,moreadvancedtechniquesareneeded.Twobasicsolutionsareavailable:1)producingthecontent
hostThefully-qualifieddomainnameoftheclient,oritsIPnumberifthenameisnotavailable.
identIfIdentityCheckisenabledandtheclientma-chinerunsidentd,thenthisistheidentityinformationreportedbytheclient.
authuserIftherequestwasforapasswordprotecteddocument,thenthisistheuseridusedintherequest.dateThedateandtimeoftherequest,informatday/month/year:hour:minute:secondzone
requestTherequestline(url)fromtheclient,enclosedindoublequotes(”).
statusThethreedigitstatuscodereturnedtotheclient.bytesThenumberofbytesintheobjectreturnedtotheclient,notincludinganyheaders.
0-7803-5643-8/99/$10.00c1999IEEENovember10-13,1999SanJuan,PuertoRico
29thASEE/IEEEFrontiersinEducationConference
12a3-19
Session12a3
dynamically(ontheserver)sothatappropriatemetadatacanbeincludedasforexamplehiddeninputfieldsetc.and2)HTTPCookies[5]thataresmallinformationfragmentstrav-elingwithHTTPheaders.
Theformermethod,producingcontentsdynamicallysu-persedesthelatter,cookies,invariousaspects.Perhapsthemostimportantoneisreliability:notallbrowserssupportcookiesatalland,eveniftheydo,theusercandeliberatelydenytheiruse.Cookiesarealsoverylimitedintheircapa-bilitiesofcarryinginformation,forinstance,thesizeofacookiecannotexceed1024bytes.Still,cookiesareverycommonandusedespeciallyfortrackingtimingorusers’pathswhenbrowsingthroughaWWWsite.
Producingthecontentsdynamicallyallowsthedeveloper(teacher)tohavealltheloginformationavailableanytimeandfurthermore,tospecifywhatmetadataandhowitisin-cludedintherealcontents.Therearedrawbackstoo:itismuchmoredifficultforadevelopertocreatedynamiccon-tentsthanstaticWWWpagesanditalwaystakesmorecom-putationalresourcesfromtheserver.
Awidevarietyofimplementationtechniquescanbeused:CGI(CommonGatewayInterface),apachemodule-techniques(modperl,modphp,etc.),JavaServletsandsoon.Also,inordertomanagelargeamountofdata,thatisCBE-materialasWWWpages,adatabaseofsomekindisneededtoo.SQLortextdatabasesseemtobethemostpop-ularchoicesatthemoment.
Itshouldbenoticedthat,inprinciple,client-sideinterac-tivitytechniquessuchasJavaAppletsorscriptinglanguagesdonotprovideloginformationfromtheprocessingitself.ThisisduetothefactthattheWWWlogsystemisbasedoninteractionbetweenclientandserverandclient-sideprocess-ingbasicallydoesnotinvolveserverforotherthanprovidingthecodeitself.
9.EXAMPLEANALYSIS
Considerthefollowing(simplified)logdata:IP1IP1IP2IP1IP2IP1IP1IP2IP1
user1user1user2user1user2user1user1user2user1
time1time2time3time4time5time6time7time8time9
url1url2url1url3url2url4url5url7url6
codecodecodecodecodecodecodecodecode
sizesizesizesizesizesizesizesizesize
ref0ref1ref0ref2ref1ref3ref4ref2ref5
brwsrbrwsrbrwsrbrwsrbrwsrbrwsrbrwsrbrwsrbrwsr
Whatistheamountoftimetheyhavespentonpage/pages?(seenotebelow)
Havetheyconnectedfromlocalnetwork,i.e.,usedforexamplecomputersprovidedbyschool/university(IP-address)?
WhatWWW-browsersoftwarehavetheyused?
Asthemselves,theresultsarenotverydescriptive,butifthesemeasuresandurlsaremappedwithrespecttotheCBEmaterialtheypresent,theycanbeofgreatvalueinlearningevaluation.Forexample,wecanobservethatuser1hasstudied6pages(topics),spentcertainamountoftimeonapage/pages,hasfollowedalinearpath,hasusedacertainWWW-browserandcomputerconnectedtouniversitylocalnetwork.User2ontheotherhandhasstudied3pagesincertaintime,hasjumpedtoanewtopicfrompage2andhasuseduniversitymodemconnections.
Furthermore,ifthereisaninteractivetaskonpage4thatrequiresapplyinginformationfrompages1-3,itwouldindi-catethatuser1hasindeedmanagedtoovercomethetaskandthuslearnedthattopic.
ItshouldbenoticedthatbecausetheWWWlog-systemisbasedonrequestprocessing,thetimingisnotaccurateincertaincases.Forinstance,usersmayjumptosomeotherwebsitebytypingintheaddressdirectly,laterreturningbacktotheoriginalpageandthejumpwouldnotbeseeninthelogs.However,inmanycasesitisreasonabletoassumethatusersstudyeducationalmaterialwithconcentration.
AnotherimportantfeatureoftheWWWlog-systemisthattheloggingisalwaysperformedontheparticularWWWservercontainingthematerial.Thismeansthatifmaterialisdistributedoverarangeofservers,thelogdataneedstobecombinedbeforeanalysisoradvancedsolutionslikeredi-rectingneedstobeused.
ThefinalnoticeregradingthisexampleisthatiftheCBEmaterialonlycontainsbasicorreferencematerial,butnotanyinteractivityortasksthatrequireapplyinginformation,thelog-basedevaluationgivesmetadatatypeinformationthatismostusefulwhencombinedwithotherevaluationmethods.Ontheotherhand,incaseofself-study,inter-activeWeb-basededucation,theloganalysiscanprovideaverygoodpictureofthestudent’slearningprocessandthusbeusedasanevaluationmeasurebyitself.
10.ADVANCEDWWWSTRUCTURE
Touseloggingfortrueinteractiveguidance,basicallyeverysinglewebpagehastobeturnedintoacallforaprogramintheserver.Onesolution(seeFigure1)forthisistouseaUserAgenttofiltertrafficbetweenclientandserver[3].(Note:UserAgenthereiscompletelydifferentfromtheonementionedinsection7).
Fromthatdata,immidiatelycanbecalculated:
Whatpageshaveuser1anduser2visited?Whatpathhavetheyfollowed?
0-7803-5643-8/99/$10.00c1999IEEENovember10-13,1999SanJuan,PuertoRico
29thASEE/IEEEFrontiersinEducationConference
12a3-20
Session12a3
ServerClient
CommunicationsServerControl + DataUserInterfaceDataUserAgentApplicationAgentestinlearningevaluation.Naturally,thesewouldbecase-dependentparametersandcertainlymoredifficulttoimple-ment.
Besidesevaluationoflearning,loggingisaveryusefulconceptinotherrelatedareasaswell.Anexamplecouldbepsychologicaltesting,whereitissometimesevenmoreim-portanttoknowhowapersontakesthetestthanwhathe/sheanswered.Creatingsuchtestinanelectronicformratherthantraditionalpaperversion,andincludinglogginginthesystem,wouldresultinanextendedsetofmaterialand,per-haps,increasedaccuracyforpsychologicalevaluation.
12.REFERENCES
[1]KarjalainenM.,RahkilaM.,“Learningsignalprocess-ingconceptsandpsychoacousticsintheQuickSigDSPenvironment”,inProc.1995IEEEInt.Conf.Acoust.,Speech,andSig.Proc.(ICASSP’95),Detroit,Michi-gan,USA,vol2,pp.1125-1128,May9-12,1995.
http://www.acoustics.hut.fi/˜mara/publications/icassp95/
ComputationEngineFigure1:AnadvancedWWWarchitectureInthisstructure,aUserAgentrepresentstherealuserandtakescareofalltherequestsmadetotheserver.Itrespondstotheserequestswithappropriatedataaccordingtospecifi-cationsmadebythedeveloper.ThusitrepresentstheteacheraswellandtheappropriateterminthiscasewouldbeaDou-bleAgent.
Thereareanumberofwaystoimplementsucharchi-tecture,buttheseareverysophisticatedandfarbeyondthescopeofthispaper.Inprinciple,thistypeofstructureallowsloggingtobeappliedinbothoff-lineevaluationandon-lineinteractivecontrol.
11.CONCLUSIONS
Comparedtotraditionalmethods,log-basedevaluationhascertainadvantages.Particularly,logswillincluderealinfor-mationofthelearningprocessitself,whichisquiteimpos-sibletoexposewithtraditionalmethodslikewrittenexam-inationsorassessmentforms.Theconceptofloggingalsosupportsthecognitive-constructivetheoryoflearning[6],providedthattheCBEmaterialitselfwasconstructedinthismanner.
Loggingalsoaddsnewpossibilitiestostudentself-evaluation,whichinuniversitylevelstudiesandstudent-orientedlearningisveryimportant.Furthermore,log-basedevaluationincludesastrongdependencyonthecontents,sameasexaminationsandfeedbackforms.Whenusedincombinationwithexaminationsorfeedbackforms,loggingcan,however,provideamoreaccurateevaluationoflearn-ing.
Furthermore,loggingdoesnothavetoberestrictedtouserandtransactionrelateddatabutmorecontentorientedpa-rameterscouldbeusedaswell.Forexamplecontentde-scriptionsorcontentlevelwouldbeparametersofinter-[2]Wiezel,A.,“Measuringthesuccessofvirtualtu-toring”,inProc.1998IEEEFrontiersinEducation(FIE’98),Tempe,Arizona,USA,Nov12-15,1998.
http://fairway.ecn.purdue.edu/˜fie/fie98/papers/1357.pdf
[3]RahkilaM.,KarjalainenM.,“AnExperimentalArchi-tectureforInteractiveWeb-basedDSPeducation,”inProc.1998IEEEInt.Conf.Acoust.,Speech,andSig.Proc.(ICASSP’98),Seattle,Washington,USA,May12-15,1998.pp.1857-1860.
http://www.acoustics.hut.fi/˜mara/publications/icassp98/
[4]Fielding,R.,etal.,“HypertextTransferProtocol–
HTTP/1.1”,RFC2068,January1997.
ftp://ftp.isi.edu/in-notes/rfc2068.txt
[5]Kristol,D.,Montulli,L.,“HTTPStateManagement
Mechanism”,RFC2109,February1997.
ftp://ftp.isi.edu/in-notes/rfc2109.txt
[6]Bruner,J.S.,“TowardaTheoryofInstruction”,Har-vardUniversityPress,Cambridge,MA,USA,1966.
0-7803-5643-8/99/$10.00c1999IEEENovember10-13,1999SanJuan,PuertoRico
29thASEE/IEEEFrontiersinEducationConference
12a3-21
因篇幅问题不能全部显示，请点此查看更多更全内容