VideoPoints

• ••• • •• • ••••• • •
COSC 4364 Numerical Methods - Lecture23_Least_Squares_Fit
Help Tour
© Distribution of this video is restricted by its owner
Transcript ×
Auto highlight
Font-size
00:00 I don't like this. You I prefer it was more like
00:09 And okay. Like one of my some like our american bad things.
00:16 then I'm not to the point where like Russia. Russia is not
00:24 I better not say something so I be over a beer or coffee.
00:39 . So switching topics. Uh now talking about the squares the aggression or
00:49 . I can fit because described the to what it is. I remember
00:55 the Yeah, I remember you have comments or questions a few lectures
01:02 So this will be the topic And the first is just talking about
01:14 the problem. That's what, what it? The machinery and then talk
01:18 little bit from other actually deal with . Um In terms of all basis
01:27 , you mentioned that when we talked polynomial interpolation and I'm talking about actual
01:35 community and there's nothing to be described the book. All right. So
01:45 gonna just recapped in some ways um about data fitting report and investors.
01:53 was an executive in the use David for function values and then try to
02:01 an exact fit assuming you have infinite or arithmetic is perfect. Then the
02:09 of the act an exact match. first to be the single polynomial and
02:15 talked about spines that this still breaking up into a collection of polynomial.
02:20 feeling a few points and then having conditions among them and it tended to
02:26 in that case that was beneficial in of the behavior between the points start
02:31 circulate by having a low order polynomial of a single polynomial. That from
02:38 point becomes very hard work. So the scores is now changing things.
02:44 now we don't want to do an match that somehow. Yeah. Bit
02:53 you pulling over or function of some to a bunch of points recently won
02:59 not necessarily exactly. So that's what course problem is all about. And
03:05 there's a few examples first to take table here. And so the underlying
03:14 in this case is that somehow for year relationship in this case between the
03:21 tension s and the temperature. So it was the perfect um linear relationship
03:29 the physical difference between successful values. all the same given that is the
03:36 difference between each one of them in various but it's not so either in
03:42 relationship isn't linear or there is a , there's always some area with some
03:50 in your system. So the actual . So even if in fact the
03:55 thing is still a linear relationship. observations may not reading prudent support.
04:03 um so then is if it is aligned vest within your relationship in this
04:10 between temperature and surface tension. Next how do you find the equation for
04:15 line? And um this is just that it doesn't need to be mistakes
04:22 . But in principle it's a so here's a picture of the table
04:30 and it may look like this. yes, it's it looks like reasonable
04:35 approximation. The linear amount wouldn't be that. So it's again trying to
04:39 it in your feet too. So now how do you decide on
04:47 their life? So one thing is much you believe is going to be
04:54 the best place, slope and some or an ox beef. Then look
05:03 the various points of observation that is the XK. And then for the
05:09 value, let's talk about. So have sex. He rejects, you
05:14 an observation of some sort. So they look that's where the distance between
05:21 the lines that are being for the XK and the action observation. So
05:28 kind of thing that doesn't seem to so bad is you look at the
05:31 error and then you try to basically some of all the absolute errors,
05:39 is one way of doing it and not necessarily a bad way at
05:44 Um and if american looks like the , so this means it's a call
05:50 one norm basically just sum up all absolute distances. It turns off,
05:57 mathematically Finding the Red A&B. And kind of L one norm, it's
06:05 the easiest thing to do. So just said, I don't want to
06:09 that by being your programming me. it's um in general and nonconference
06:15 It's No, so that's why it out that that's not the most common
06:23 of doing it. So instead what commonly done is to try to seek
06:29 minimize the square of their instead of the sum of the absolute values.
06:36 of course, if you work for square of the errors to come,
06:40 more bigger because it's square supposed to . So it has a different results
06:47 different ways and beans, if you this criteria versus the everyone know some
06:54 them why people still do this is want to actually penalize biggers more than
07:03 , but not that this makes sense it turns out that if you try
07:07 minimize this thing, you can analytically out what A and B should
07:12 So it's It's definitely preferred one. also the case um that if the
07:24 , suppose that is a linear relationship real, but your observations, yK
07:30 not quite following the line. But the reason that doesn't follow the line
07:36 that there is normally distributed errors for arrows in why, then it turns
07:45 that this formulation also gives you kind the best possible estimates of A but
07:52 only yes, this is true and was just and jesus defense yesterday and
08:02 and the students confused this thing where actually terrorists. So custom it
08:14 So using the squares of this criteria minimizing things doesn't cause the error to
08:23 an abnormal distribution. It entirely depends what it is and they no,
08:29 that's 1000. So anyone. So is I think we can look at
08:34 norms is also the Euclidean or else norm. But you may also choose
08:40 . Holmes said the L one norm sometimes preferred, but it's hard to
08:46 . And I can also use fire no one's like it distracted. But
08:54 the book and my lectures, they stay with to these squares At the
09:00 is based on the two norm. least squares comes from about.
09:07 so as I said, one can an analytical solution in the usual
09:10 They want to find an extreme point to take the derivative with respect to
09:15 variables that you can manipulate in these their coefficients or Yeah, for the
09:21 line, the slope, the offsets A and B. To take the
09:27 of this respect for A and And you take this case is something
09:33 easy. There's the chain rule, two comes down. They have
09:37 It is and then you have the of what's inside the parenthesis respect for
09:42 and B. So if you take derivative respect to A then what's left
09:47 XK So that comes outside and it's relatively respect to be well the derivative
09:55 the respective itself is just the So it's kind of an interesting one
10:00 for that. So this is derivative to A and the derivative respect the
10:05 now we have two operations into So now they can figure out what
10:10 they are by Sullivan the system of . So um, the number two
10:17 is kind of irrelevant because we have on the right hand side. So
10:22 can just divide by Tuesday and it make any change on the right hand
10:26 . He just forgets about # And then you can sort of rewrite
10:31 expression. So the two unknowns are and B and Y. 0.
10:36 K. is the observations. So the YK and XK they're known entities
10:42 stick them on the, on the hand side. Similar thing on this
10:47 on the bottom here, we just the summer Y K. To the
10:50 hand side. So now I'm gonna a little two x 2 system that
10:55 can use them to solve for. And this system, whether it's a
11:00 x 2 or much larger system. this whole thing is known as the
11:06 equations that comes off of the principal minimizing the sum of squares. So
11:16 questions is something you should definitely remember they are not to construct. Absolutely
11:24 in the open in terms of these and fine example the amount of excess
11:31 corresponding wise and trying to fit a line. Um so this was what
11:36 derived for this straight time sitting today . And now here we have the
11:41 . Case and the white case when plug it all in and these submissions
11:46 then it turns out this is what get in terms of any questions.
11:50 all it's pretty small system. So can solve a thing. We get
11:54 . And B. And now you an equation for the straight line fit
11:58 this set of table values and it like this. So it's kind of
12:04 . It's a pretty good fit looking . And that this again, so
12:10 you look at the deviation in between observation and wide value for the line
12:18 some of all the squares here then turns out this is the best fit
12:23 the sense that minimizes the sum of of their distances and then the
12:27 And so that's kind of the principle this first. And the questions from
12:35 principles and how your derided so making little bit more complicated is just doing
12:47 more formally perhaps. So I understand long term try to hit to keep
12:56 of what's ah some align values in case A. Y. And what's
13:03 actual observations are going to use? predicted or estimated values to have their
13:09 squiggle fellow and then why is the measurement or observation. So this is
13:17 the L two norm for the That's the to unravel it it's the
13:22 of squares obvious to unravel the Investment. It's my transposed, that's
13:29 vector. So this is something. back to transit common vector and become
13:35 sum of the squares and in the . So if you want to minimize
13:41 , you know formally affect the derivative this fellow and work it all
13:46 And what is this question? So is now the normal equation in symbolic
13:52 or general form. Doesn't matter what in the rows and columns. Studies
13:57 a but this is the point. all of it I assume. So
14:06 is again and it will come back time today. There's not a lot
14:10 questions and I'll say it now and say it many times their memory comments
14:18 talk to the item values and saying their composition. Normally you can solve
14:23 thing that was done on the previous . You know, computing this 80
14:29 matrix that was on the previous concrete and then solve it. But usually
14:37 about condition numbers. So our aim not so great condition or condition Condition
14:45 for 80 K is much worse. in that case you have it's difficult
14:52 get good accuracy and directed work and come back for that. So this
15:04 since the previous example was too the equations or a experts be and try
15:12 hit that. Oh but it doesn't to be very much anything you like
15:21 for a first approximation what you want the is that the expression should be
15:30 . And they're called christians A B C. Ah how the relationship between
15:38 and X. It can be not there is this example, but they
15:43 things to be a linear combination of else. And it's this sort of
15:48 function to do that in a combination . So if you have this form
15:55 expression um where there's a nonlinear relationship X and Y. But it's a
16:01 relationship between A B and C. right. So if you do this
16:06 proceed as we did before you write now the error and this here is
16:12 kind of predictive thing using this equation the proper avian species and then the
16:19 and then square. And then you on and take the derivative respect of
16:23 B and C. With respect to . Because a B and C are
16:28 variables to try to find the best for. So if you do
16:33 then, and I'll get something a bit more ugly. The it's the
16:39 thing that basically the two comes down put it away already here because all
16:45 . And then you have the parenthesis and then it's the derivative respect for
16:50 in the first equation here. So , it's the parenthesis times Ln
16:57 Because that's the derivative with respect to . You will see Ln X in
17:01 the places. Um and one because the first term. So this is
17:08 on the outside of this thing you get Ln x Ln X coming from
17:13 right. And you see it in first on the right hand side to
17:17 moving to the next to the right side. Similar things for the other
17:21 . So now, because we have parameters, A, B and C
17:26 got the best for the 3.3 matrix for the that is now effectively implicitly
17:33 explicitly the A T A matrix. it's basically and then you can so
17:39 you have A B and C and can make that column vectors. It
17:42 to have a matrix times column vector , B and C. And then
17:47 have the right hand side that was on this side story is a bunch
17:55 values and attracted to the fitting this between by an ex And talking all
18:04 this. Uh they have on the and two this matrix coefficients. And
18:11 you get something like this and then can solve it and we got to
18:16 . So you can see and then can plug it in. So now
18:21 is under than the current that is best least squares fit to the
18:29 So in that case you get something looks at this again, the relationship
18:33 X and Y. It's not playing but it was linear respect to the
18:38 . Abc. So in this case is a part of what they
18:44 If you use to do the squares using this type of relationship with him
18:57 straightforward. So These two examples I one was 59 And the other one
19:05 the 5th. He spoke to every . So in general, um you
19:12 said and many things you can choose there's a question. So choose a
19:22 ah functions you want to used for combination in terms of doing the splits
19:32 I will talk about this size But before that, just make the
19:37 here that going from this type of For linear one symbolically called this functions
19:46 for the moment. So that's the have a collection of functions G.
19:51 you do the linear combination of those uh produce an estimate of the new
19:59 observation warrants. Yes. There should been in school year. Sorry about
20:03 . Um maybe. Yeah, I'm . But so um and this can
20:12 an important about that polynomial but it also be, as I mentioned with
20:17 times 2nd medical functions. So in couple of lectures, our thoughts about
20:24 drugs functions here and then it becomes serious expansion what we have talked about
20:31 . But that's coming in the lecture basically you can choose the G is
20:39 . And the arbitrary but the consequences there are consequences of how to use
20:44 . So, and we'll talk a bit about different choices of T
20:50 some good and some not so So yeah, these are the basis
20:57 and now you want um so just things in and just showing here
21:03 this is the projected value of And of course the difference here is
21:12 error and the someone's great error Now, in general setting we have
21:17 is still the fitting is respect to the coefficients in this linear combination.
21:26 , functions D That should be sort basis functions for the space, but
21:33 linear combination, think the river, is something magic and mystery the same
21:38 before. But not formally, you the studies type of inner products.
21:44 is inner some here is respectful. , so it is um going through
21:53 various um so in this case you in to change the formation order of
22:00 . So they compute ah than We have one particular question.
22:09 And then so this becomes now product you do thank you for giving All
22:17 . And the succession of the Jason moving through the columns. So
22:23 has become something A in their So, if you look back at
22:31 concrete example of this, I can hear the sum is over the points
22:39 points. Xk and that's fine about swing Well, seriously, the inner
22:46 has gone through the exclamation points or observation point, sorry and formed the
22:55 and then they bring in the And so this is indeed again The
23:00 Matrix. And that's where the 80 matrix that went on in this particular
23:07 . This is one of the row . And that moreover with each other
23:12 to change. They are that is from the so it can be difficult
23:20 join us and our for completely. now a little bit tired to choose
23:25 Gs and we'll talk about that. They want us to be linearly independent
23:32 otherwise you can just reduce the problem if anyone of them can reform the
23:37 combination or the other genes, that one of them is not necessary.
23:43 you can do with so yeah, would be proper to the problem at
23:53 . Remember in a few months when talked about interpolation um yeah this
23:59 Right. So your function or table . It's a drug function. It
24:05 strict function as a basis function. on the name one that is exactly
24:12 function we're looking at. However, you use the mono me als you
24:16 , the exact square likes to Germany a whole lot of them to the
24:21 that sign of a call center. um that's what's hidden behind this issue
24:27 the appropriate. So the closer the of these basis functions or to whatever
24:34 trying to approximate the fewer of the lady. And ah they should
24:47 in the condition normal equations. So against Yeah. So here this that
24:55 are the matrix values. So this of matrix values that results from evaluating
25:03 things should because some metrics that you to be well conditioned or things to
25:11 never numerically well, is it salt intraday accuracy? And in particular,
25:20 these basis functions are orthogonal, the normal operations over the A.
25:28 A becomes the identity matrix. So come back to that. But that's
25:37 it means. Now if you look this particular something here, the only
25:43 that is non zero is when I J. R. The same.
25:47 when the indices subsequent here are If these basis functions are orthogonal,
25:54 means ah yeah, they promised Yes. Yeah, there's a so
26:03 . So that's how I tried to uh these uh basis functions. So
26:09 are they should be linearly independent. a stronger thing is that they are
26:15 this case for cardinal and they're not . They want at least uh the
26:23 a matrix to be as well conditions possible. So how you choose the
26:28 will affect in the world in this , you never think I'll show that
26:36 on the soul. But this is best decision. If there are any
26:40 independent, then there are the basis the vector space. So that means
26:46 solution values. Do you just fine this business functions best is the space
26:56 is kind of span by these places . Um and I'll come back to
27:03 as well in terms of kind of it geometric illustration. So let's
27:13 So here is not coming back again what these things is. So as
27:17 said if there are normal Then there's thing is only one if it's or
27:25 normal course it's octagonal. That is zero. Only one of these
27:29 This is all the same and it's top of that. The normal some
27:33 the best the length one electors for than it's just the one thing.
27:42 if they are best of the matrix generate to the identity matrix north and
27:49 solve this directive from the question. one each row in the matrix is
27:55 the diagonal entry on the left hand and whatever it is on the right
28:01 . So and there is no division here because it is Ortho normal.
28:06 diana and all right. So so , you know this is just the
28:19 . Um No if disease are not talking about to each other. Um
28:26 we can use its punishment process that talked about that you left us back
28:31 make them with our girl and then them so that you get on a
28:36 basis but many times did not go this trouble. But it's just a
28:43 of constructing the north the normal their active. Yeah. Normally questions this
28:49 the song. Any questions so All right. So one thing is
29:05 like one convicted mono meals. Um we're still on let's uh form as
29:14 basis function. So it's just sort one basis functions. Another one X
29:19 . That's perform it in your combination the basis functions that are X.
29:25 then that means approximation after seeking is of this form. Now if you
29:33 these things in and form the normal , you got this blank checks.
29:39 is the random on budgets. And thought about it and it's also assignments
29:46 playing around with another movement. And is it doesn't take much of A
29:53 and before this matrix gets very in . So using the memorials. That's
30:03 racist. But for during function approximation . It's not records. And that
30:15 the same when I talked about for normal Exactly the anomalous or not basis
30:24 no different when it comes to be . So here it was found that
30:31 I showed before where? But the they are best Toby type of
30:38 Whereas the other ones suggest cause of , the mono meals. And that
30:44 of the reason is that um Even they are different in some ways the
30:51 of the chorus of X. It's similar. And so the standard interval
30:55 this point. All right, everyone's quite different so that it's easier to
31:05 that's a better condition number or the Gs naturally or in some ways now
31:14 all. But they are at least more significant in the sense, in
31:19 directions. So projections onto the it is easier to get accurate than
31:26 protection. So warriors coming more or in the sense that this is not
31:33 . So championship is a group versus also my constituting these points.
31:45 so the next step of sides is going to do the exercise um using
31:52 uh basis functions for doing these So I mean he has just temperature
32:04 . They're constructed in this way. there and the officials before. So
32:09 is the first in formula get serious successive evaluations of it also means they're
32:20 deficient because if you want you have a range of calling all meals.
32:27 , there's some approximation is a linear of basis functions. Right? So
32:34 if they want to add one more book? one more basis function.
32:39 you for a given X value. have already evaluated these two months or
32:45 get And other basis function in And the amount of work we need
32:49 do is very little So you have two function values already known and we
32:56 to do a couple of multiplies and add and then we get for a
33:01 function are bigger than that. You also use this expression but computational these
33:08 kind of afternoon whereas decided to do I would say when users just use
33:13 type of formulation of it instead of place question. So so now here's
33:26 gun now and they're living their combination . That's the form, the approximation
33:31 something based on the set of basis . Now the 80 a matrix and
33:38 normal questions. Then the entries in matrix are based on The Chef Michelle
33:45 No one else generally that means a better condition matrix than the then if
33:54 has figured out and the solution to equation. So to solve this equation
34:02 you get your cds and then we evaluate duties, putting in the seas
34:06 different cities and what is the first the book this place to set
34:12 Yeah, official polynomial. So it out that the relatively simple to find
34:19 evaluation and it's best to kind of Ryker shin from the highest end that
34:28 choose to be included and basis sets functions and then they go down and
34:36 in the end conditions in fact the of G of X. That is
34:44 least squares fit to whatever nature And the next second science is going
34:54 you that this in fact this is that this is and the radiation but
34:58 have to go through a bunch of . But computational they are quite
35:03 So in the accept of steps that's for playing around this expression for
35:10 . O. X. This was young linear combination of the basis
35:14 And see you can get out of equation. There's no these two terms
35:19 the other side, plug it in , proceed and then there's a whole
35:24 . So manipulations are ah This right our first break up into the three
35:31 and then play it on with the here and get the summations ah
35:40 He um Well, it might be obvious here, but it comes to
35:45 other side of the island. Change range here and start shopping zero then
35:51 for conservation it tends to be good . Change the range. And these
35:59 . So now I feel blessed to the same W. J. In
36:04 of the sounds here and then you advantage of the things that Uh this
36:15 . And so this one is the one. Um Now, Well to
36:20 next one here, the last one plus one you can basically have the
36:26 . W. N. Plus one Starting value was zero. So that's
36:31 this one that's the plus one And similar here for the past two
36:36 because so now at least you have same upper submission but you have a
36:43 um So we're starting index and reporting that. And now I'm trying to
36:51 the same summation range. There's a of terms through here and one year
36:56 makes the call out and then they something like this and then and so
37:05 the summation. You know, these , that's my not just here and
37:10 there are the three explanations and it like this. This is exactly
37:18 the records in formula here. So is actually zero. So what's left
37:23 essentially this thing and then yes, , at least consolidate these two ghosts
37:34 this is what you got. So best thing was going through once I
37:40 . Indeed, we can evaluate the approximation in terms of the basis function
37:48 continuing the Ws and this one. then I think, uh, well
37:57 is the standard thing, I guess contract many times that's true with you
38:01 . And so the championship was almost on the -1-1 interval. No wonder
38:07 not what you have. So there just variable substitution is you can use
38:13 . That's for me. One is one to want any arbitrary control of
38:18 . Can I be interval after the is one part. So no single
38:24 banks hopefully. So this is And so here is based on what
38:30 have done in terms of what the actually is. Here was the gender
38:39 to have an arbitrary interval A B the microphone and then it's it's performing
38:47 normal equations here and the next time that shows how you can do this
38:53 then you have to solve these things then you get overseas and the army
38:58 will eventually computer these between first is how we evaluate and Fungal 80 a
39:10 championship polynomial for an argument. And call it Tjk. And there is
39:17 of observation points X O C K this case of this repression formula.
39:24 yes, you have two of them it's easier to compute the next
39:28 And that's what you basically see Ah all the investment table if you
39:35 of the values the matrix, That's not quite the 88 matrix but
39:41 have everything you need from this matrix the A. And so um this
39:54 basically computing the various Interest in the a matrix in this case it's a
40:00 matrix and all that. So I the amazing are the same Mr
40:07 a product that is um in the the interest here. Yeah. It's
40:17 much and don't commit after this And this first officials using temperature polynomial
40:28 various degrees then what the confirmation of collection points points front over. I
40:42 questions how it works the Turkish economy of its simple recursive structure seem to
40:52 and the normal matrix is well we so that's why it's good. So
41:03 they are pulling our meals but they constructed in a particular way so that
41:07 are nice properties confidentially and american Yes, it's pretty good. And
41:21 question, Yes. So max so kind of a progression from at some
41:35 . That's been one month before progression talk today lecture. It's best
41:46 start with the ***. The ships linear equation. That may not be
41:51 good. And then he said, , next thing is to use straightforward
41:57 the Newsman Amiel. Unfortunately, that to generate normal equations that trail
42:04 So that's not to be recommended. we have the championship that actually gives
42:09 good normal equations and that's fairly easy compute. Um, now, as
42:17 mentioned in terms of juice investors if the basis functions also have the
42:25 , that was the north, then trivial to solve the normal equations because
42:33 88 matrixes diagrams. So next they taken the book has a way of
42:44 acceptable, but probably not for, know, so for that first schools
42:53 angle documentation for this inner product or Various entries in the 18 Metrics provided
43:02 the right hand side. So this just a normal property on it.
43:10 , that doesn't matter. That's Think of it as vectors and it's
43:15 row vector times column vector and It matter which one in the product as
43:24 with itself, invest the sum of elements and positive on the several
43:33 Um, we're scaling properly on that lingering in the arguments we'll use that
43:43 terms of now manipulating and questions. I will make some comments to this
43:53 and to make that clear and what motivation is for these formulas. Um
44:01 came after that. This set upon normals constructed in this particular way is
44:12 it's kind of incursion here where you three different problems. You have the
44:22 uh and the previous one to generate next polynomial. And as you can
44:29 from this formulas, of course it's constant simple one. And this is
44:34 first degree polynomial. And then what see here in this expression, The
44:41 of the polynomial increases by one for and that you use because here's the
44:49 increases by one, but the rest it is basically what the degree was
44:56 . So the polynomial degree increases by and then the often get us our
45:03 . And in this particular way to sure that these polynomial are what's
45:13 Ah And but you know the next we can show with this alphas and
45:23 , I'm sure that things falling on R. And D. What's up
45:32 contenders. So you've got these 2 and what you are watching products public
45:38 our unravel it If you want to out where the choice of α
45:45 That was on the previous slide this after this year. You can do
45:51 next 1-2 versus 2. 1. you should also be a zero.
45:58 is working through again what the definition Q. Two and Q one is
46:04 it in. Unraveling it and plugging the of zero and 01. And
46:09 turns out it's also safer. Um , the one thing I think Next
46:18 is Q. three horses, zero the next side. And ah no
46:37 . Mhm. Um So, I first of all, they are not
46:42 they had, wow. The thing that this cannot be zero.
46:47 this could be in a product or with itself against um cannot be
46:52 And trying to convince us about and just looking at them. So this
46:59 a product of the paranormal to and itself. Um And there's support for
47:05 argument here that if that zero second many rules to be anything except The
47:14 that is zero. So, That that's the reason still to show that
47:23 comes to zero and 0. I think that was something that's fine.
47:29 hopefully um I don't believe that. right. So, in that
47:37 they are at least Pataki in all then 6 7 in the previous slides
47:43 are um Now and then there's serious that keep going um to show that
47:49 are in the in fact I'm starting so there since they're rhetorical, their
48:01 basis for a vector space um that can express and a function in that
48:12 I was putting all male some combination this separate. So that's sincere that
48:22 now is a linear combination of the polynomial that um And then they gave
48:35 stepped forward in any way of figuring what the proficiency or that you know
48:39 polynomial that construct according to the lower that was on the previous side.
48:48 you can have the cues um than best compare african rights and side for
48:54 correspondent power acts now and in this just start for the highest order ah
49:02 of X. And it's only the that has the highest or X.
49:08 . In it. So the degree the queues are equal to the
49:13 So and the power of X. actual polynomial. I knew basically the
49:21 for that power of S. And . Yes. Then also the ah
49:30 eh em for Q. I. in that case and then you can
49:37 . Okay fine. I have taken of the highest of return And then
49:41 can subtract it. So now you an N -1 degree on the left
49:45 side and -1 degree on the right side. And I do the same
49:50 of and the coefficients for what's on left and right hand side. So
49:56 we get the next stage or else another side to do uh or
50:05 This is I guess what I said down. So. And then this
50:10 what I was about to say about products on the doctor outside. And
50:16 the cures are orthogonal, the only on the right hand side is when
50:22 if you then the product with Stevie . That's the only thing that survives
50:27 the right hand side Is the term QJ zero also invested. This is
50:35 way of thinking about public education. that singing in front of quick as
50:43 in terms of So Yeah, they're pull a no mess. And I
50:52 one is the functions that I will about that. But the benefit is
51:01 of clients is that now there are . So that means the 18 and
51:10 diagrams. Not necessarily. But then of that, we also need to
51:16 the cues. And even if you get the diagonal matrix and it's,
51:21 it? That's all? Mm I have another question. So,
51:27 conclude with this illustration, The useful one is the championship as the basis
51:34 . And what are these um ah is partially basis functions. You think
51:45 is the construct? I'll come back using this particular basis functions and the
51:55 topic. So I came back to heart. You choose. I said
52:04 how you choose the key, ideally should choose them. So there are
52:11 of their properties or what you're trying approximate. So you don't need many
52:17 them to get a good thing. only one their spines and so
52:25 And oh for example in the traps doing trig functions for another treat function
52:36 that they don't manage maybe another There was also talked about stds and
52:43 up a bit and you know, within this processing example finding that you
52:48 not need mm hmm terms. And expression conducted a singular values and which
52:59 hello, sufficiently small. Nothing but the response to the north. So
53:08 how do you come to choose? that had the same amount of their
53:12 ? We could choose basically singular vectors on how significant they think the values
53:21 in terms of the number of actors needed. So now I'm going to
53:27 talk about the similar things in terms the scores fits. So. How
53:33 of the basis functions do you need have for a sufficiently good approximation?
53:40 really And that sense of that's where about the machinery and making sure that
53:50 normal equations are well behaved. It necessarily tell you anything. How many
53:57 functions do I need around talk about . It's more straight questions here.
54:08 again, if we have now the functions generally g they quit the publisher
54:12 it can be either issues that were normal, no more problems that we
54:20 talked about and then the console there's . This is the kind of polynomial
54:27 the approximation and that with the design the combination of the basis function.
54:36 what the d squared method was doing trying to minimize this expression here.
54:45 the jews the seas to find this function such that darkness wherein sum of
54:53 areas meaning. And this particular expression is also known as a very so
55:06 , so this is come on kind criteria so um for the rest here
55:12 of forgetting about this pediatrician because the that as most impacts is that so
55:22 um many basis functions you kind of to um get this approximation of this
55:30 so forever an increase and then your that this variance will decrease and at
55:37 point the variance is small enough to this is acceptable. And then the
55:44 is, what is that then? many basis functions do you need to
55:49 the segment? That is small And so this as well if the
55:56 behind the wise is in the polynomial this at some point when the degree
56:03 your approximately in polynomial is equal to underlying form. Otherwise it's the same
56:12 to get to the point where this this variance 1,000° animals. So in
56:22 way and said here you can do and error. You start with a
56:26 , you increase the number of basis and as long as they are decreases
56:31 then they keep going at some point said that now that doesn't seem to
56:36 much. So I'm ok with what is. That's what they're doing in
56:41 book. Done is to find it clever way instead of doing it.
56:48 Trial and error and step by step and tried to figure out what degree
56:56 or what degree basis functions through action need to get the desirable um for
57:02 violence. So I just kind of and so no, they didn't approximation
57:13 is the finger combination of enough to with the cues because the reason is
57:20 survivorship these guys are targ and also simplifies expression. So that's why rewrites
57:29 it used to be before. No cuba instead of the G a second
57:35 . And then I remember that now kinds of our diagonal respect to each
57:40 and rewriting kind of inner product notation the inner some here is in the
57:46 as he did in the practice. this looks a lot cleaner. And
57:51 these guys were octagonal, the only that survives on the left hand side
57:59 the least two industries are the So that means from gets and ci
58:07 . I'm sorry, divided by the entries in on the left. That's
58:14 . So it's really easy to find and there are probably independent and so
58:22 can compute disease. Um depending on many what the degree of the polynomial
58:29 how many basis functions want. But of the seeds that have computed would
58:36 if you decided to have one more of the two basis functions. And
58:43 reason is because the queues are a so the production's onto the existing set
58:48 change when you're sort of one more in your space. So that's now
58:55 the next five. I wanted. , simplicity and borrowing your size to
59:01 instead of like a. You will . Yeah. So so basically we
59:12 on the um that's what I'm Yeah, I'm sorry about that.
59:18 discovered too late my best if you this is what went on to work
59:25 . Um So now I'm going to from white to the errors are basically
59:33 . Um PMS. This is kind the error and we looked at the
59:36 relative to the various basis functions and this thing then so this is a
59:48 relationship because of the in a product . So they can keep going here
59:55 . Now is this expression the way was to find, I was seeing
60:01 product with this polynomial but because of functionality, the only thing you get
60:07 is then discard and then you want to be kind of a cardinals for
60:16 . Again you have to pick it but the sea was from the previous
60:20 . Sorry um things for this. the sea was substitute F. Here
60:28 this fine. So then um this essentially. Exactly And this divided by
60:35 . So that's why. And ensure mhm error in this approximation is
60:44 Anyone of the mhm basis functions show picture our products come to market.
60:56 then again, so now this is thing. You kind of I want
61:04 be small enough at some point that has the approximation is good enough and
61:10 variants Francisco. But focus on minimizing . And now we look at
61:16 This is basically square girl. Hello. And in here and then
61:24 have to see how and get the formula for there's some more squares and
61:37 have this thing. So let's see the next. So investors are here
61:43 best if we want this one to strong enough they can they have the
61:50 the U. N. So we generate these polynomial. So we know
61:54 in the product. Always with the . We can have this other set
62:00 the vines. They come together and this in their products so that they
62:05 . This food is very simple recursive instead of trying the full polynomial things
62:11 out or not. The signal for small enough. And that's what that's
62:19 exercise what's about to figure out what sort of choose. And that can
62:27 simple Nepal good heart Children. Yeah instantly. Oh So this is what
62:39 said so far said I will give 10 metric and she's not where's the
62:45 ? That's the sure thing from trying activate this function. Um, Championship
62:54 numbers are okay forward the normal questions still this thing is orthogonal basis functions
63:02 there was this particular way of generating answer have orthogonal basis functions. And
63:10 that was also used to show how can decide how many for that particular
63:14 functions that you need to get the level of approximation or an acceptable level
63:23 b squared. So yeah, I'm to tell you. And actually they
63:33 complete the scores. So again this for the congress. He said The
63:40 two norm has chosen normally because it's form an analytic expression for the
63:47 We need to solve the final That gives a good approximation numbers.
63:56 now your country. So um go and talk about the matrix formulation
64:10 So I don't know if you have G's. Ah it wasn't. And
64:16 your combination for all the different observation but perhaps um tried to determine the
64:26 . Such the squared error between from device and the corresponding as predicted,
64:37 there are minimal amenities for insects. writing it down then in terms of
64:49 . Um, matrix form another So stuff in the matrix and here is
64:53 set of unknowns. The c vector years is something protected by values dr
65:01 are these days and see to be close as possible to devise into these
65:10 . So one thing to notice they have a bunch of observations and
65:19 are whatever number of basis functions, So general, this matrix is not
65:27 square matrix. Um So this is A matrix not the A.
65:32 A. So a number of Just a number of observations and
65:40 And the number of columns is the of basis functions which is to
65:48 So and I actually want number on sticker off like they did before and
65:55 comes off taking the derivative of Um figuring out what the equation is
66:01 too. Um That minimizes. So way of looking at that is kind
66:10 Yes, four of them from the is what this H and C.
66:16 is basically than within your combinations of different basis function as the news.
66:22 in some ways this linear combination then ST connector. And the space spanned
66:31 these forks for this ah basis Um And the city's filling their combination
66:40 this dysfunction. So that means eight see is confined to be in this
66:50 defined by the basis functions now. what that means, what we're trying
66:59 do is and the squares if you at it. Picture space or geometric
67:06 is they want to find the linger or the the sector. And the
67:17 spanned by the columns of a that closest to life. And that means
67:23 in some sense the different area director to be orthogonal to the spacefaring.
67:33 to me that's sort of one way building at what the discourse approximation
67:39 it finds the closest point. Um that's that's the error from that
67:46 There's so far enough to this So and that's just trying to point
67:52 here that this just come out anything . The rest of the seas are
67:59 all these things and the error that so cool subspace uh is common.
68:05 that means for every column. um the error is again the inner
68:12 is zero because the error, there's cardinal to each one of them.
68:18 I would have a projection onto one them and you could the estimates.
68:24 so this is true for all the this is another way of the the
68:31 equation. But now doing kind of geometric interpretation of the the squares.
68:43 now a couple of pictures, all and they want to polynomial. So
68:52 it is difficult what to do to this whole time. And then they
68:57 a polynomial approximation to the surface And then they want again the error
69:02 be orthogonal. So that's the Try to take the polynomial in this
69:08 . I guess two of the variables try to take it out. Uh
69:12 is someone can have this type of and and then let's analyze this fellow
69:23 . So through these like surface store dimensional you can have more dimensions of
69:29 and you look at The error two and minimize it. So this is
69:35 of what to do and computational geometry absolutely to this thing. And early
69:47 when I started the doctor as these shoulder this drama movie and up and
69:56 ah slam or simultaneous localization and I'm sorry. S fifties the figure
70:05 that and now I'm going to come that most precisely. But before that
70:12 I don't know. And then back the normal questions just say very little
70:17 this but a little bit more about here rather normal question on using the
70:28 interpretation. So I think I'll skip this particular directs over when I talked
70:37 so indirect systems. So let's get kind of clever version of got some
70:46 when you have a Selectric matrix like . So it allows it to the
70:51 of the work. But in principle contain yes, it can attack this
70:56 the direct solvers like johnson elimination or that symmetry. But again this is
71:03 condition so that tends to normally or . So you don't really want to
71:08 that so many times to find So the next thing that doesn't suffer
71:14 the air conditioning is apparently Is the were polemic say never formed 88.
71:22 then what you can do, you use householder transformations, the characterization on
71:29 nature and that I come up from talk about I am values and inflation
71:36 also saying we evaluate their composition. householder transformation generates few matrix and is
71:44 rectangular matrix that has one column for other Matrix eight columns. And the
71:50 of the queue matrix is that isn't of the normal meetings, so to
71:58 . You is your internet matrix. what is your factor A. And
72:03 cuba nor our Sylvester they have on left hand side of the factory session
72:08 our times the unknown that could see wide and then multiply from the left
72:17 . So then the que teacher and and George and she is the
72:20 T. Q. Is one service you can see on the left hand
72:24 , in Q. T. On the right hand side. And
72:27 it's all this triangular system of equations the are inverse. But it's a
72:33 tricky assaulted lee the triangular questions. so in this case you never formed
72:39 A. T. A. So don't worsen the condition number for the
72:43 A. And we can just proceed and officially think differently than it
72:50 So that's For number two on approach three because to use singular right in
72:57 composition some of that case form. it's not computing STD is not
73:05 So it has that drawback the war finally chief. Um But nevertheless the
73:13 also has its advantages. So and there's so much to actually go ahead
73:18 do that. And then that's just reminder of the properties of the therapy
73:22 seems however under their composition. So I'll see you there and this
73:28 So just know that final output. so he was really why this kind
73:36 singular value decomposition works and how they themselves. So this is again,
73:40 is for his problem that you want find the except minimizes this expression.
73:49 . And it's a manipulator. Start advantage of duty. Thank you.
73:53 the identity matrix. And stick it here. And so these two things
73:58 the same because of the properties and D. A. And can unravel
74:03 by writing this thing and sending the . And you will see the do
74:08 shows up in the middle and you financed the natures and you propagate the
74:13 into this depression. So now you these two guys uh then we can
74:18 do again read and be transported because the template matrix. So that looks
74:24 nonsense at the moment. But Than rest but have duty eight times
74:32 And looking at the single value of composition of A. That means uh
74:39 expression is in practice the singular value signal. So and so that's the
74:48 again. So the result of now at this thing in practice basically this
74:55 the only thing you need to worry is z. That is this
75:03 Mm hmm. And then because sigma matrix, what you end up having
75:09 this expression here And it says are assuming that some of the singular values
75:16 zero. So basically itself being a for this part only is going through
75:24 first are Elements correspondent, a non singular values. And then you see
75:30 still what it is. So then have the remainder proceed square where there's
75:35 stigma zero. So um thanks. now did you get something here?
75:44 can um To minimize this thing. are invested two and Z is related
75:51 X. So we want to minimize . This is the constant with respect
75:55 X. So you want to minimize expression And it's not very simple.
76:01 eventually you can get all the correspondence for the approximation that you have done
76:08 terms of or find disease rather than have the singular values from the composition
76:15 A. So the you can find . Because we came from the single
76:23 in their composition and I have a . Uh, and then it's
76:30 and then to also get what the squared error is in terms of,
76:38 , sort of the remaining square. you have a way of getting um
76:44 politicians and the approximation or the So we have exactly one. And
76:53 also can find things. There is a function of the number of single
76:58 lives in a non-0. So I that uh, so not nervous some
77:05 . That's the summarizing what they what the new square system it is
77:11 a function of the body we need find first and there is an example
77:23 fitness matrix single by their composition. this is the full value that they
77:32 . They're just zeroes here contribution And then okay. I think the
77:43 approximation from this physical matrix. This what? Yes. In congress.
77:51 stop. So again minimized the sum predators. Um, be careful to
78:02 a basis functions that needs to the matrix A. That's the first
78:09 And practically never form a ta Do some other methods to work.
78:17 speaking approximation equations. A times War characterization or symptomatic composition. So
78:29 book again tried to stress forming normal and solving them. Okay hmm.
78:44 about using to celebrate this. Thank you us. Mm
-
+