- June purpose scientists devise diverse variables and observe their variations. However, those variables which are hypothetically chosen for the sake

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI Article Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) 1 ITAKI, Masahiko Contents Introduction Partial correction of orbit analysis Some features of principal component analysis Analysis of Japan s GDP 1 Basic analysis 2 Orbit analysis 3 Principal component analysis 1 Terms 2 Interpretation 4 Developmental principal component analysis (The following is in the next issue.) 1 Two-principal-component analysis: extraction of basic opposing relations 2 Three-principal-component analysis: extraction of developed opposing relations 3 Four-principal-component analysis: extraction of opposing relations in totality (a hierarchy of opposing relations) 5 Principal component analysis and orbit analysis combined 1 Multiple regression analysis of principal components 2 The multiplier effect measured by multiple regression analysis of principal components 3 Orbit analysis of principal components 4 Orbit analysis of variables and principal components: extraction of causality Key words : GDP, Multiplier effect, Orbit analysis, Developmental principal component analysis, Multiple regression analysis of principal components, Orbit analysis of variables and principal components, Causality Introduction Either in natural science or social science, the fundamental purpose of research is to understand changes of an object over time on the basis of its structural analysis. For that

- June purpose scientists devise diverse variables and observe their variations. However, those variables which are hypothetically chosen for the sake of measurement of variations rarely represent the actual substance of the object or forces that bring about its changes. We could expect much less to identify clear causality among those variables. It is the case for aggregates of GDP that are taken as an example in the present article: a bit surprising as it may sound to the common sense of Keynesian economics, those aggregates, i.e. fixed capital formation, household consumption, trade balance and government consumption, are all variables for observing and measuring a national economy, rather than its substance or driving forces per se. In order to understand this point, let us take an example of school examinations of five subjects, i.e. literature, mathematics, natural science, social science and English. Although they are five subjects for educating students or variables for measuring their academic achievements, intrinsic academic capabilities can be more properly explained by certain synthetic variables independent from each other, such as general academic ability, natural-science-biased ability and humanities-biased ability. This is the basic idea by which principal component analysis is conducted. The purpose of this article is, as already suggested above, to examine the question of What is the unit of change? The unit can be understood as substance of a research object or a force that brings about changes, which we analyze on the basis of ordinary principal component analysis. It is followed by developmental principal component analysis in order to elucidate multilayered structure of forces that bring about changes, in which orthogonal principal components are dialectically interpreted as multilayered structure of unified opposites. Conducting orbit analysis for leading-following relations (Itaki (2014)) among all variables and principal components, we construct a new theory of causality in which principal components are causes and variables are results. It is a replacement of so-called Granger causality that was rejected in principle (ibid). In addition as a byproduct of using GDP for our example, a precise method of measuring the multiplier effect is proposed with the help of multiple regression analysis of principal components: i.e. an increment of GDP by means of one unit increase in fixed capital formation, household consumption, trade balance or government consumption. I. Partial correction of orbit analysis Orbit analysis will be partially corrected because some of the Microsoft EXCEL s functions in V. Calculation of the direction of orbit rotation and leading-following relations, Itaki (2014), give either wrong results or inconsistent treatments when three coordinates necessary to determine the direction of orbit rotation are in the following three cases: 1 Cases in which line ab is horizontal: if three coordinates are, for example, a(1, 0) b(2, 0) and c(3, 1), angle bac is correctly calculated to be positive, i.e. an anticlockwise rotation, and the result is correctly that variable x leads (X). However, if three coordinates are, for example, a(1, 0), b(2, 0) and c(3, -1), angle bac is correctly calculated to be negative, i.e. a clockwise rotation, but the result is wrongly that

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI variable y leads (YY); it should be that variable x leads (X). 2 Cases in which line abc is horizontal: when angle bac is either 0, or, the direction of rotation should be unidentified ( ), but the results are wrongly that variable x leads for and variable y leads for. 3 Inconsistent treatments in various cases in which coordinates become vertical or horizontal are corrected: when three coordinates a, b and c become vertical, #DIV/0! comes out and when they become horizontal, #ERROR comes out, because their leading-following relations are unidentified (although when only two coordinates become vertical or horizontal, their leading-following relations can be identified. Refer to the illustrations below for details). In those cases, one ranking point should be divided between the two variables by 0.5 and 0.5. Corrected functions are as follows in the same format as that in Itaki (2014). The shadows are corrections. Note that even after the corrections the global system of short-term interest rates in Itaki (2014) does not change. Slope: D3 =SLOPE(C2:C3,B2:B3), D4 =SLOPE(C3:C4,B3:B4) Rotation (in radians): E3 =IF(AND(-PI()<=IF(0<=ATAN2(B4-B2,C4-C2),ATAN2(B4-B2,C4- C2),2*PI()+ATAN2(B4-B2,C4-C2))-IF(0<=ATAN2(B3-B2,C3-C2),ATAN2(B3-B2,C3- C2),2*PI()+ATAN2(B3-B2,C3-C2)),IF(0<=ATAN2(B4-B2,C4-C2),ATAN2(B4-B2,C4- C2),2*PI()+ATAN2(B4-B2,C4-C2))-IF(0<=ATAN2(B3-B2,C3-C2),ATAN2(B3-B2,C3- C2),2*PI()+ATAN2(B3-B2,C3-C2))<=PI()),IF(0<=ATAN2(B4-B2,C4-C2),ATAN2(B4- B2,C4-C2),2*PI()+ATAN2(B4-B2,C4-C2))-IF(0<=ATAN2(B3-B2,C3-C2),ATAN2(B3- B2,C3-C2),2*PI()+ATAN2(B3-B2,C3-C2)),IF(PI()<IF(0<=ATAN2(B4-B2,C4- C2),ATAN2(B4-B2,C4-C2),2*PI()+ATAN2(B4-B2,C4-C2))-IF(0<=ATAN2(B3-B2,C3- C2),ATAN2(B3-B2,C3-C2),2*PI()+ATAN2(B3-B2,C3-C2)),IF(0<=ATAN2(B4-B2,C4- C2),ATAN2(B4-B2,C4-C2),2*PI()+ATAN2(B4-B2,C4-C2))-IF(0<=ATAN2(B3-B2,C3- C2),ATAN2(B3-B2,C3-C2),2*PI()+ATAN2(B3-B2,C3-C2))- 2*PI(),2*PI()+IF(0<=ATAN2(B4-B2,C4-C2),ATAN2(B4-B2,C4-C2),2*PI()+ATAN2(B4- B2,C4-C2))-IF(0<=ATAN2(B3-B2,C3-C2),ATAN2(B3-B2,C3-C2),2*PI()+ATAN2(B3- B2,C3-C2)))) Rotation (in degrees): F3 =DEGREES(E3) Leading-following: G3 =IF(AND(ISERROR(D3),F3<0,F3<>-180),"YY",IF(AND(ISERROR(D3),0<F3,180< >F3),"-YY",IF(AND(0<=D3,0<F3,180<>F3),"X",IF(AND(0<D3,F3<0,F3<>-180),"YY",IF (AND(D3<0,0<F3,180<>F3),"-YY",IF(AND(D3<=0,F3<0,F3<>-180),"-X","-")))))) Dissolved functions for rotation: H3 =B3-B2 H4 =B4-B2 I3 =C3-C2 I4 =C4-C2 J3 =ATAN2(H3,I3) J4 =ATAN2(H4,I4)

- June K3 =IF(0<=J3,J3,2*PI()+J3) K4 =IF(0<=J4,J4,2*PI()+J4) L3=IF(AND(-PI()<=K4-K3,K4-K3<=PI()),K4-K3,IF(PI()<K4-K3,K4-K3-2*PI(),2*PI()+K4-K3)) Table 1: Calculation of the direction of orbit rotation and leading-following relations in EXCEL YY leading a c b c b a Xleading b a YY leading a Xleading b c c c b #ERROR a b c X leading a X leading a b c c YY leading b a a a c Xleading Xleading b a #DIV/0! a b b a a c YY leading YY leading b c b c c c b YY leading

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI II. Some features of principal component analysis It would be almost needless to say that we always witness in our research extremely complicated phenomena unfolding themselves in front of us and seeking explanations. So we devise certain variables (or indicators), with which we attempt to describe and understand the phenomena in terms of relations between the variables (or indicators). Here we often encounter a troublesome dilemma: if the number of variables is small, our understanding of the phenomena remains unsatisfactory; to the contrary, a large number of variables would hinder their clear-cut understanding. One possible solution to the dilemma is to limit the number of variables to an appropriate one from a certain viewpoint. However, if we can sum up information that a large number of variables offer and turn them into a concise number of representative synthetic variables (or comprehensive indicators), our understanding of the phenomena could improve to a great extent. The principal component analysis has developed on this basic idea. H. Hotelling, one of the original developers of principal component analysis, talked about its significance as follows (Hotelling (1933), p.1): Consider n variables attaching to each individual of a population. These statistical variables x 1, x 2,..., x n, might for example be scores made by school children in tests of speed and skill in solving arithmetical problems or in reading; or they might be various physical properties of telephone poles, or the rates of exchange among various currencies. The x's will ordinarily be correlated. It is natural to ask whether some more fundamental set of independent variables exists, perhaps fewer in number than the x's, which determine the values the x's will take. (Hotelling (1933) p.1.) Hotelling seems to distinguish two types, i.e. a set of more fundamental and independent variables and a set of fewer variables in number. The distinction between them will play an important role later when we proceed from ordinary principal component analysis to developmental principal component analysis. I. T. Jolliffe, who has been fascinated by principal component analysis for over 30 years, mentioned with this respect as follows: The central idea of principal component analysis (PCA) is to reduce the dimensionality of a data set consisting of a large number of interrelated variables, while retaining as much as possible of the variation present in the data set. This is achieved by transforming to a new set of variables, the principal components (PCs), which are uncorrelated, and which are ordered so that the first few retain most of the variation present in all of the original variables. (Jolliffe (2002) p.1.) N principal components that are obtained from n variables have the following properties : 1 N principal components that are obtained by linearly combining original n variables are uncorrelated to each other: i.e. principal components are orthogonal to each other in n-dimensional space. 2 The following inequality relations hold among the principal components: the variance of the first principal component > the variance of the second principal component > > the variance of the nth principal component. The total variance of all the principal components is equal to that of the original n variables. The ratio of a variance of each principal component to the total variance is called a proportion. A few top principal

- June components can often explain much variance of all variables. Firstly, an example of principal component analysis on non-time-series data is examined in order to understand some of its important features: i.e. examinations of 50 students in literature, mathematics, natural science, social science and English. We start our analysis with 50-students 5-subject scores and their total, the objective of which is to measure their academic ability. Those 50 students are ranked in order of their total scores. A simple observation with respect to total scores would reveal a variety of patterns of academic ability. Some students are very good at literature and some other students are very bad at natural science. There are some very talented students who gain high scores in all subjects, but others are less talented and get miserable scores in all subjects. These facts may suggest that the score of a specific subject, such as English or mathematics, does not correctly predict variations of total scores, and that simple total scores are not a perfect indicator of general academic ability of students. Generally speaking, however, top students are likely to acquire high scores in any subjects; in the middle, there are some characteristic students who are extremely good, for example, at English or incredibly bad, for example, at mathematics; and students in the bottom are likely to gain low scores in many subjects. Our experiences would suggest a hypothesis that academic ability could be measured by a certain combination of subjects instead of independently by a single subject: for instance, general academic ability may exist which exerts positive effects on all subjects. The more general academic ability increases, the higher scores of mathematics and English as well as literature and others can be expected. Alternatively, natural-science-biased ability may exist: it has a positive correlation with scores of natural science and mathematics, but a negative correlation with English and literature. By contrast, we may think of humanities-biased ability, which might be a reversed one to natural-science-biased ability and thus, we should regard them as heads and tails of the same coin, i.e. natural-science- or humanitiesbiased ability altogether. On top of that, there may be a specific ability that has a high positive correlation only with mathematics, but has nothing to do with all the other subjects, which could be named mathematics-biased ability. A crucial point is that those general academic ability, natural-science- or humanitiesbiased ability and mathematics-biased ability are independent from each other and have no relation with each other. Ability that is only and specifically biased to natural science or humanities is different in dimension from comprehensive ability with no bias to any specific subject. That is also true to ability that is concentrated only on mathematics. It would be reasonable to assume that those components of academic ability with different dimensions scatter over 50 students and each student has them all in various proportions. By comparison, 5 subjects, i.e. literature, mathematics, natural science, social science and English, are independently carried out in examinations, whose scores more or less synchronize together thanks to overlapping effects of those components of academic ability. Therefore, those 5 different subjects are categories of items for measuring academic ability, rather than categories of substance that form academic ability.

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI The relations, in our example, between subjects and components of academic ability are equivalent to those between variables and principal components. By means of principal component analysis, complicated variations in scores of all subjects are resolved into a set of independent components of academic ability, and in turn, by means of multiple regression analysis of principal components, scores of each subject and their totals are reconstructed by those academic components. Next, imagine the stage of a kind of psychodrama, in which tense emotions are exchanged among 10 actors and actresses, in order to understand some features of principal component analysis on time-series data: Six out of ten of them belong to the largest group of friends; remaining four form another group of friends. Those two groups are in conflict, but two in the minor group keep some relations with members of the major group. The reason for the conflict is that the bosses of the two groups are rivals to one another. There are some factions in the major group that exercise delicate maneuvers. A woman in the major group and a man in the minor group are lovers, just like Juliet in the House of Capulet and Romeo in the House of Montague. We are already informed of those complicated human relations among them, but the audience is not. And, furthermore, the play is a pantomime and the audience has to understand those entangled relations only with the help of men s and women s actions on the stage. Now, the pantomime begins. Members of the major group start to gather in the center of the stage. But, their movements are not straight: its factions keep some tactful distances among them while assembling. Those two members in the minor group who keep somewhat friendly relations with the major group exhibit rather complicated movements: although they are mainly attracted by the minor group that gathers at an edge of the stage, they also make eyes at the major group and a certain faction as well. They are intermingled in those triple human relations. Romeo and Juliet may show the most complicated maneuvers of all. Juliet shifts towards the center of the stage, where her own major group gathers, and Romeo towards the edge, both with reluctant steps. They actually would like to spend a good time together without being bothered by others at the opposite side of the stage. They gradually come close together, but quickly separate and temporarily keep some awkward distance when someone of the opposite group approaches. Those triple human relations are reflected on their highly complex movements: i.e. the conflict between the two groups, minor conflicts among factions and love between themselves. The reason for difficulties to comprehend the plot of the play is that one actor or actress embodies not only one human relation, but also several intertwined relations, which altogether determine each movement on the stage of an actor or actress. What would happen then, if those multilayered human relations are respectively separated from individuals, brought together and classified by human relation? Attraction of the major group, minor group, factions and lovers, and repulsion between the major and minor groups, among factions those are all perfectly independent forces from each other. Human relations with respect to the major group probably work as the strongest force of all, and those with respect to the minor group and others would be ranked according to the

- June strength of their attraction. Those invisible, mutually independent human relations, after being separated, could be expressed concretely as movements on the stage: the attraction of the major group would go straight to the center of the stage, that of the minor group to the edge of the stage and that of the two lovers to the opposite side of the stage, which are all independent from and not interfering with others. Those ten actors and actresses are categories that demonstrate bearers of human relations changing over time, rather than their real substances. As exemplified by the pantomime above, the actors and actresses who combine and bear a variety of human relations are equivalent to variables, and human relations that are respectively abstracted and put together are equivalent to principal components. We later show that variables complicated variations over time are resolved into a set of independent principal components and, by means of multiple regression analysis of principal components, the original variables are reconstructed by the principal components. Now we proceed to explore a frontier of time-series multivariate analysis with an example of Japan s GDP, in which firstly some important features of principal component analysis are concretely demonstrated, secondly it is further developed into developmental principal component analysis and lastly it is combined with orbit analysis. III. Analysis of Japan s GDP The outline of our argument is as follows: (1) Basic analysis and (2) Orbit analysis deal with, as it were, the visible world of observers, in which we observers would be able to directly confirm, only if paying appropriate attentions to data, such statistical phenomena as value of change, rate of change and direction of orbit rotation. By contrast, (3) Principal component analysis and (4) Developmental principal component analysis deal with the invisible world, in which principal components abstracted from data could not be detected at all with our bare eyes. It is a world of concept that we can reach only as a result of certain statistical procedures. Lastly in (5) Principal component analysis and orbit analysis combined, those visible world and invisible world will be combined together, in which we understand how the world observers see with their bare eyes is constructed and operated by invisible forces behind the scene (1) Basic analysis Table 2 Japan s GDP reveals historical development of Japan s GDP from 1956 to 2012: i.e. annual changes in value at current prices and annual growth rates, instead of nominal or real absolute value of GDP. The reason is that the analytical purpose here is to capture the driving forces of change in GDP, rather than its long-term trend. Its graph is Fig. 1 Japan s GDP (annual change in value at current prices), which illustrates main four aggregates of GDP, i.e. household consumption, government consumption, trade balance and fixed capital formation. The reason why we adopt nominal values that are influenced by changes in prices due to inflation or deflation is that we later perform orbit analysis on those data: annual nominal values are

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI Talbe 2: Japan's GDP Data IMF, International Financial Statistics. indispensable for examining actual pulling and being-pulled relations in quantity among those aggregates. Household consumption, the center pole of an economy, indicates that the Japanese economy could be divided into three periods: one until the early 1970s of stable growth, another in the 1970s and 1980s of rapid expansion and the last one since 1990 of shrinkage owing to the burst of an economic babble. Figures 2, 3 and 4 Japan s GDP (annual growth rate) illustrate annual growth rates of those aggregates. Comparisons between household consumption and government consumption, between GDP and fixed capital formation and between GDP and trade balance make clear their particular features: household consumption and government consumption move closely along with GDP; the movement of fixed capital formation amplifies that of GDP; and trade balance, by and large, negatively correlates to GDP. Observation of changes in value and rates of change allows us to get much insight into

- June Unit: million yen 20,000,000 Household consump Government 15,000,000 consump Trade balance 10,000,000 5,000,000 Fixed capital forma n 0-5,000,000-10,000,000-15,000,000 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 Fig. 1: Japan's GDP (annual change in value at current prices) 30.0 25.0 20.0 15.0 Household consump Government consump GDP 10.0 5.0 0.0-5.0-10.0 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 Fig. 2: Japan's GDP (annual growth rate) 30.0 25.0 20.0 15.0 GDP Fixed capital forma n 10.0 5.0 0.0-5.0-10.0-15.0 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 Fig. 3: Japan's GDP (annual growth rate)

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI 30.0 25.0 20.0 15.0 10.0 5.0 0.0-5.0 GDP Trade balance 400.0 200.0 0.0-200.0-400.0-10.0 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012-600.0 Note: Trade balance is on the right axis. Fig. 4: Japan's GDP (annual growth rate) variations of GDP. There are, however, two main obstacles or new problems in such basic analysis: Firstly, those four aggregates are neither independent from each other nor move autonomously; they are rather causes and effects of each other and thus, move in a bunch, being entangled with each other. Our new task would be, therefore, to disentangle the unity and synchronism among the aggregates, and to identify leading and following variables. Secondly, the synchronization implies, for example, that household consumption is not independent and is under the influence of government consumption, trade balance and fixed capital formation altogether. Even when you observe and describe specific variations of household consumption, you actually observe synthetic effects of those four aggregates through the window of household consumption. This holds true for government consumption, trade balance and fixed capital formation as well and thus, even if you conduct four individual observations, you actually observe the same phenomenon four times in succession. This fact suggests that those four aggregates are four items necessary for measuring GDP, but they are not four categories necessary for explaining variations of GDP. We have to identify more essential and autonomous substance, forces or motive power that is hidden under those four synchronizing items. It should be emphasized, however, that basic statistical information, such as annual changes in value and annual growth rates, is the goal as well as the starting point of our analysis. We will come back to the basic information after overcoming those two limitations and solving the problems; then, it will be accompanied by rich additional information and shed a new explanatory light onto the structure and movement of Japan s GDP.

- June (2) Orbit analysis Orbit analysis is a statistical method that extracts leading-following relations between two variables by combining coordinates of time-series data along time and calculating the slope and the direction of rotation of the orbit thus depicted. In the case of many variables, calculations of all combinations between pair variables produce a hierarchy of leadingfollowing relations among all variables (Itaki(2014)), which identifies the kick-starter variable that heralds all variations among other variables and determines their order of following. The method is applied to the four aggregates of Japan s GDP in the same period in Fig. 5 9-year moving average of ranking points for Japan s GDP (expenditure). Due to wide annual variance of ranking points, 9-year moving average is adopted out of 5-, 7- and 9-year calculations, which seems to represent most appropriately medium-term variations. Annual changes in value are used for the calculations, because there have to be quantitative pulling- and being-pulled-relations among variables for them to be actually leading-following relations (Itaki (2014) p.16, pp.30-31.). Household consumption, government consumption, trade balance and fixed capital formation are apparently in quantitative pulling- and being-pulled-relations in their annual changes in value. That is not a priori true in their annual growth rates: although an increase in household consumption by 10 billion yen, for example, directly causes quantitative deterioration in trade balance or an increase in fixed capital formation by 1 billion yen or so, its increase by 5 %, for example, does not necessarily cause an increase or decrease in trade balance or fixed capital formation by a certain percentage point. Fig. 5 shows that the kick-starter (i.e. the first leading variable) was fixed capital formation in 1958-67, household consumption in 1972-80, government consumption in the period of fiscal reconstruction 1983-85, fixed capital formation again in the period of an economic bubble and its burst 1988-1992 and trade balance in the period of the great depression 1996-2009. Another feature is that fixed capital formation was in the lowest rank in 1977-84, the period between the aftermath of the first worldwide recession since 3.0 2.5 Investment Household c Government c Bubble Trade surplus 2.0 1.5 Household consump on Government consump on 1.0 Trade balance 0.5 Turning point Turning point Fixed capital forma on 0.0 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 Fig. 5: 9-year moving average of ranking points for Japan s GDP (expenditure)

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI the end of Second World War, 1974-75, and the previous year of the Plaza Accord in 1985; and that happened again since 2000. Investment in plants and equipments does not now play the role of generating changes in GDP. On top of that, two major structural transformations of Japanese economy took place, first in the end of the 1960s and early 1970s and second in the first half of the 1990s in the aftermath of the burst of bubble, when ranks were intertwined in a bunch and changed quickly. We now know that information acquired by orbit analysis by and large corresponds to our widely shared knowledge about the post-world War history of Japanese economy, and also that it adds many other insights to our understanding. We ought to pay enough attention to the difference between the two concepts, leading and following and preceding and lagging. The unit of our observation and analysis is one period of time, the beginning and end of which we can observe and compare, but we do not know actions and reactions that may take place among variables during the minimum one period of time. Therefore, we cannot know, in principle, which variable precedes or lags, or whether reversals in ranks repeat themselves, during the unit period, just like in a black box. What we can actually observe and measure is distinction between a variable that actively leads changes and a variable that follows the changes initiated. Those changes are produced by complicated actions and reactions (positive and negative feedbacks) among variables. The distinction is observed and recorded as if being temporal preceding-lagging relations. We should be careful enough, therefore, that leading-following relations in Fig. 5 among four aggregates are not temporal preceding-lagging relations. Their distinction reveals its importance later when orbit analysis is conducted among the principal components of those four aggregates. An application of orbit analysis to annual variations in value in Fig. 1 exposes hidden leading-following relations among the aggregates in Fig. 5. It is certainly a big step forwards to the understanding of structure and movements of the object, which, however, touches only its surface; we next proceed to principal component analysis and further to developmental principal component analysis that allow us to investigate more essential and autonomous substance, forces or motive power of the object that is hidden under the surface of those leading-following aggregates. (3) Principal component analysis 1. Terms The results of principal component analysis on the four aggregates of Japan s GDP are shown in Table 3-1 Eigenvectors and loadings (Japan s GDP) and Table 4 Principal component scores of Japan s GDP. Explanation of terms in these tables and some features of principal component analysis are given as follows: Principal component analysis on the basis of variance and of correlation: there are two types of principal component analysis, one that is based on original data, i.e. their variance, and another on their correlation. The latter is conducted on standardized scores (i.e. Z scores) whose standard deviation is one. Here we adopt the former because the four aggregates share the same monetary unit, i.e. the Japanese

- June Table 3-1: Eigenvectors and loadings (Japan's GDP) Table 3-2: Square of loadings (Japan's GDP) yen, and we would like to conduct orbit analysis and principal component analysis on the basis of monetary value. The latter is usually used in the case in which variables have different units of measurement or in which, despite sharing the same unit, difference in value does not make sense for one reason or another. Principal components: four principal components are extracted from four variables. With the first principal component being Z 1, household consumption X 1, the government consumption X 2, trade balance X 3 and fixed capital formation X 4, we get Z 1 = 0.692 X 1 + 0.097 X 2 0.042 X 3 + 0.714 X 4 from the eigenvector in the table: the first principal component is composed as a synthetic function of the original four variables. So are the other principal components. Eigenvector: eigenvectors that consist of the coefficients of the principal components have a property that the sum of squares of all elements in each row and each column is one: for example, regarding household consumption 0.692 2 + 0.508 2 + 0.476 2 + ( 0.193) 2 = 1, and regarding the first principal component 0.692 2 + 0.097 2 + ( 0.042) 2 + 0.714 2 = 1. Values and signs of elements give us an important clue when we attempt to interpret the meanings of principal components. Loading: the sum of squares of all loadings of the first principal component, i.e. household consumption 4,683,135, government consumption 65,502, trade balance

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI Table 4: Principal component scores of Japan's GDP (unit: 1 million yen) 287,146 and fixed capital formation 4,833,451 is equal to the eigenvalue 45,805,510,321,072. This holds true for the second principal component and others. Loadings represent composition of an eigenvalue of each principal component and give us another important clue, in addition to eigenvectors, when we attempt to interpret the meanings of principal components. Eigenvalue and its proportion: the sum of eigenvalues of all principal components is equal to the variance of the original variables, which is shown in Table 3-2 Square of loadings (Japan s GDP) : all information about the variance of the original variables is turned into that of the principal components. As in Table 3-1, the

- June eigenvalue of the first principal component is the largest and that of the fourth principal component is the smallest. The share of the eigenvalue of each principal component in the total eigenvalue is shown as its proportion: the first principal component contains 71.4% of variance of those four variables, the second 15.7%, the third 8.1% and the fourth 4.8% in the decreasing order. Therefore, the first and second principal components with 87.1% in total can explain most of the variance of all the variables. Principal component analysis has the function of effectively summarizing the number of variables. Generally speaking, the higher the correlations among the original variables are, the larger the proportion of the first principal component is and the more quickly the proportions of the second and other principal components decrease. To the contrary, when the correlations are low, the proportion of the first principal component becomes small and those of the second and others decrease slowly. Principal component analysis on the basis of correlation of variables with no correlations with each other will produce the same proportions among principal components: for example, 0.2 each for five principal components. Principal component scores: they are time-series values of each principal component that are calculated by means of coefficients in an eigenvector (see Table 4 Principal component scores of Japan s GDP ). They are usually calculated with their mean being zero; scores with their mean not being zero are also calculated in the table. Correlations among variables and principal components: Table 5 Correlation matrix (Japan s GDP) provides correlation coefficients between variables, between variables and principal components and between principal components. As stated above, all principal components are orthogonal to each other and thus, their correlation coefficients are all zero. By contrast, in the case of principal component analysis on the basis of correlation, the order of correlation coefficients between variables and principal components is the same as that of coefficients in an eigenvector, and their correlation coefficients are the same as the loadings. In the case of principal component analysis on the basis of variance, however, neither property holds true. The original four variables, i.e. household consumption X 1, government consumption X 2, trade balance X 3 and fixed capital formation X 4 are not orthogonal and thus, more or less correlated to each other. Suppose there are four axes in the four dimensional space, each of which represents each variable, they Table 5: Correlation matrix (Japan's GDP)

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI are not orthogonal and their variations influence each other. Four principal components are synthetic variables, Z 1, Z 2, Z 3 and Z 4, made of the four variables X 1, X 2, X 3 and X 4, and set to be orthogonal in the space with their correlations being zero. Orthogonal relations are extremely useful for regression analysis. If we conduct multiple regression analysis with X 1, X 2, X 3 and X 4 being independent variables, a serious problem of multiple colinearity always occurs owing to mutual correlations among them. It is a rather peculiar phenomenon in which signs of some coefficients of those four variables derived from multiple regression analysis may be reversed against our theoretical expectation or in which reliability on some coefficients may substantially go down. Unfortunately, the trouble does not seem to attract enough attention in some empirical researches. However, multiple regression analysis with principal components, rather than the original variables, being independent variables, will make our research free from multiple colinearity and furthermore, reduce the number of variables despite achieving better results. Orthogonal and non-orthogonal relations raise a rather radical question about methodology of multivariate analysis: in our example, those four aggregates are four items necessary to measure time-series variations of GDP, but they are not appropriate categories that represent four substances of GDP necessary to explain its variations. For example, four items are more or less correlated to each other and thus, you cannot observe or describe variations of household consumption either individually or independently from others, because its variations result from comprehensive effects of all four items. This holds true for government consumption, trade balance and fixed capital formation as well; even if you conduct four individual observations, you actually observe the same phenomenon four times in succession. More generally speaking, that would lead to the question on what units should be set to correctly measure, describe and explain variations of a research object. In our example, those four independent principal components are units of four dimensions: it is just like to adopt units of four independent dimensions, such as weight, volume, temperature and position, to record variations of an object. Only after properly setting units of variations, we can search essential and autonomous substance, forces and motive power of our research object. 2. Interpretation Theoretically and empirically strict definitions are already given to GDP s four aggregates, household consumption X 1, government consumption X 2, trade balance X 3 and fixed capital formation X 4. How about those four principal components derived from the aggregates? To begin with, how can we understand the meaning of the first principal component Z 1 = 0.692 X 1 + 0.097 X 2 0.042 X 3 + 0.714 X 4? Interpretation has to be performed on the basis of signs, values and proportions of the four coefficients. I. T. Jolliffe stated about the

- June possibility of successful interpretation as follows: It must be emphasized that although in many examples the PCs can be readily interpreted, this is by no means universally true. There is no reason, a priori, why a mathematically derived linear function of the original variables (which is what the PCs are) should have a simple interpretation. It is remarkable how often it seems to be possible to interpret the first few PCs, though it is probable that some interpretations owe a lot to the analyst s ingenuity and imagination. Careful thought should go into any interpretation and, at an earlier stage, into the choice of variables and whether to transform them. In some circumstances, transformation of variables before analysis may improve the chances of a simple interpretation. Conversely, the arbitrary inclusion of logarithms, powers, ratios, etc., of the original variables can make it unlikely that any simple interpretation will be found. (Jolliffe (2002) p.64.) An outstanding feature of the first principal component is as shown in Table 3-1 that the coefficients of fixed capital formation and household consumption are as large as 0.714 and 0.692 respectively with substantially small government consumption 0.097 and trade balance 0.042. Our interpretation, therefore, should start with the fact that fixed capital formation and household consumption have positive coefficients of a similar size. The first principal component, which explain 71.4% of all variations of those four aggregates of Japan s GDP in 1956-2012, seems to represent the basic business cycle in which when fixed capital formation increases, household consumption also increases and vice versa. The basic economic process can smoothly proceed without support of government consumption or reliance on trade surplus; trade balance even has a small negative coefficient. The first principal component can be safely interpreted as a representative indicator of normal economic circulation of Japan. Fig. 6 First principal component (1) and Fig. 7 First principal component (2) examine time-series data sets of fixed capital formation and household consumption multiplied by each eigenvector and their total, which is compared with the first principal component (mean 0). In Fig. 6 fixed capital formation and household consumption similarly fluctuate because of their high correlation coefficient 0.719; their total amplifies their fluctuations. As shown in Fig. 7, the total almost perfectly overlaps the first principal component. The first principal component provides us with much information about Japanese economy. It entered a period of big turmoil after a smooth expansion until the 1970s, mainly due to large fluctuations of fixed capital formation. Stable household consumption in the 1970s made it possible, however, to keep a generally high level. This seems to be the background in Fig. 5 of household consumption being the kickstarter in the period. The Japanese economy entered a period of stagnation in the first half of the 1980s after experiencing the second Oil Shock in 1979, and a period of the unprecedented bubble from 1987 to 1990. Although the bubble was characterized by speculation in stocks and real estate, it appeared as part of a normal business cycle as a result of cumulative effects of fixed capital formation and household consumption as long as GDP is concerned. Further interestingly, the cumulative effects were preceded by

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI Unit: million yen 25,000,000 20,000,000 15,000,000 10,000,000 5,000,000 0-5,000,000-10,000,000-15,000,000 Household consump n eigenvector Fixed capital form eigenvector Total -20,000,000 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 Fig. 6: First principal component (1) Unit: million yen 25,000,000 20,000,000 Total First PC (mean 0) 15,000,000 10,000,000 5,000,000 0-5,000,000-10,000,000-15,000,000-20,000,000 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 Fig. 7: First principal component (2) fixed capital formation, followed by a sharp increase in household consumption and finally, reversed by fixed capital formation in the collapse: it is a typical case of boom and burst. The process reflects the fact in Fig. 5 that fixed capital formation after a long pause took the role of the kick-starter in the bubble period. Since the financial crisis of 1998, a normal business cycle completely died down and shrinkage of fixed capital formation dragged the Japanese economy down to the bottom, although household consumption barely supported it. As of 2012 since the Lehman Brothers Shock in September 2008, it began to recover the basic business cycle again.

- June The outstanding feature of the second principal component in Table 3-1 is the overwhelming coefficient 0.725 of trade balance in the eigenvector with positive household consumption 0.508 and negative fixed capital formation 0.459. Let us disregard small government consumption 0.079 and interpret the second principal component as follows: The second principal component, which explains 15.7% of all variations of four GDP aggregates, increases as trade balance increases and fixed capital formation decreases, which are actually negatively correlated with each other with correlation coefficient 0.198. It apparently represents formation of surplus capital due to shrinkage of national economy and its external emission: i.e. abnormal economic shrinkage and formation of international surplus capital (Itaki (2006) Ch. 6). As to the relation between fixed capital formation and household consumption, they are in highly positive correlation by 0.719 and thus, fluctuate almost simultaneously. In the eigenvector of the second principal component, however, household consumption takes the positive coefficient 0.508, although fixed capital formation takes the negative coefficient 0.459; the loading of household consumption amounts to 1,611,578 and that of fixed capital formation 1,458,606, slightly less than the former. These results suggest that in the period, in which a decrease in fixed capital formation leads to shrinkage of GDP as a whole, domestic surplus capital, emitted externally as trade surplus, sustains household consumption to some extent. Fig. 8 Second principal component (1) and Fig. 9 Second principal component (2) illustrate in time-series what we discussed above. Fixed capital formation and household consumption are multiplied by respective coefficients in the eigenvector and the results are summed up; the total, trade balance multiplied by its coefficient and the second principal component are compared with each other in Fig. 9. Addition of trade balance to household consumption and fixed capital formation by and large amplifies variations of trade balance in the same direction; in fact, the second principal component and trade balance are in highly positive correlation by 0.823. The peak years of the second principal component, i.e. 1975, 83, 92, 98, 2002 and 10, exactly correspond to those in which surplus capital filled the domestic economy. In 1975, 83 and 92, in particular, the second principal component reached its peaks despite decreases in trade surplus; it suggests that even in the case in which the existence of surplus capital does not explicitly appear owing to various noises such as mobilization of counteracting fiscal policy, the second principal component could work as a good indicator of its existence. In other words, we can interpret the second principal component as the representative that concentrates all the variations of those four aggregates as long as surplus capital is concerned. The feature of the third principal component in Table 3-1 is that household consumption takes coefficient 0.476 in its eigenvector; by contrast, those of trade balance and fixed capital formation are 0.685 and 0.525 respectively. The sign of trade balance turns negative here, though positive in the second principal component. We should note that the combination in the eigenvector of household consumption 0.476, trade balance 0.685 and fixed capital formation 0.525 stands for exactly the same as the sign-reversed

Time-Series Multivariate Analysis by Orbit Analysis and Principal Component Analysis Combined (1) ITAKI Unit: million yen 10,000,000 8,000,000 Household consump n eigenvector Fixed capital igenvector Total 6,000,000 4,000,000 2,000,000 0-2,000,000-4,000,000-6,000,000-8,000,000 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 Fig. 8: Second principal component (1) Unit: million yen 8,000,000 6,000,000 4,000,000 2,000,000 0-2,000,000-4,000,000 Trade balance eigenvector -6,000,000 Second PC (mean 0) Total -8,000,000 1956 1958 1960 1962 1964 1966 1968 1970 1972 1974 1976 1978 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 Fig. 9: Second principal component (2) combination of household consumption 0.476, trade balance 0.685 and fixed capital formation 0.525; in one case the component increases in positive value and in the other it does in negative value. Now, let us temporarily disregard government consumption with relatively small coefficient 0.172 and start our interpretation of the third principal component. Despite its small proportion 8.1%, the third principal component carries a great significance. As in Table 5, household consumption and fixed capital formation are in the highest positive correlation among those four aggregates; nevertheless, they show opposite signs to each other in the eigenvector. How can we interpret it? And although household consumption and trade balance are also in slightly