The theory of the Senate
The Senate was regarded by the framers of the Australian Constitution both as essential to the federal system and as the essentially federalist feature of the Constitution. It was an institution which defined the system of government as a federal system, and without it the system would not merit that description.
There are two essentials equal representation in the Senate and for that body practically co-ordinate power with the House of Representatives. All those who recognise what are the essentials to a true union will admit these essentials. 
.... I venture to think that no one will dispute the fact that in a federation, properly so called, the federal senate must be a powerful house .... We are to have two houses of parliament each chosen by the same electors .... 
The theory underlying the bicameral structure of the federal Parliament was that having two houses, one representing the people voting as a whole and one representing equally the people voting in their respective states, would require a double majority for the passage of laws. This arrangement applies to the passage of ordinary laws a similar formula as is applied to the passage of constitutional alterations in section 128 of the Constitution. A constitutional alteration requires a majority of the people as a whole and majorities in a majority of states. An ordinary law requires a majority of the representatives of the people as a whole and a majority of the representatives of the people of a majority of states. It is impossible for a majority to be formed from the representatives of only a minority of states. This theory was explicitly stated by the founders as the basis of the structure of the legislature:
....it is accepted as a fundamental rule of the Federation that the law shall not be altered without the consent of the majority of the people, and also of a majority of the States, both speaking by their representatives ... 
....the great principle which is an essential, I think, to Federation that the two Houses should represent the people truly, and should have co-ordinate powers. They should represent the people in two groups. One should represent the people grouped as a whole, and the other should represent them as grouped in the states. Of course majorities must rule, for there would be no possible good government without majorities ruling, but I do not think the majority in South Australia should be governed by the majority in Victoria, or in New South Wales. .... If we wish to defend and perpetuate the doctrine of the rule of majorities, we must guard against the possibility of this occurring. 
The "fundamental rule" and "great principle" has not been substantively affected by the representation of the territories in the Senate, which has only slightly complicated the mathematics. It is still impossible to form a majority from a minority of states.
This theory and this legislative structure were, of course, not inventions of the Australian founders, but were adopted from the work of their American predecessors, which had emerged as a result of the great compromise at the Philadelphia convention in 1787. That convention had also invented the federal system as we now know it, its distinguishing feature being two levels of government each directly representing the people and each legislating with direct effect on the people in its own sphere of competence. It is understandable, therefore, that the Australian founders should have regarded the federal system and equal representation of the states in the Senate as virtually synonymous.
The concept of federalism is associated with a geographical division of power, and this is the basis of American and Australian federalism. There are other possible bases for a federal system in the generic sense. For example, a federal arrangement could be based on different ethnic groups within a society. In such a system, the aim would still be to ensure that laws were not enacted without a double majority, but the second majority would be a majority of the ethnic groups rather than a majority of the geographical states. This theoretical construction of federalism is important because it illustrates the aim of a federal system: to ensure the formation of a distributed majority, that is, a majority distributed across the groups forming the units of the federation and not confined to a minority of those groups. It is not the aim of the system to have the constituent groups of the federation voting as blocs. In the case of a federation based on ethnic groups, that would clearly defeat the whole purpose of the federation, as the groups might as well form their own separate nation-states (but this is difficult where the ethnic groups are geographically mixed: the problem with which we have become so familiar since the collapse of the Soviet Union and Yugoslavia). On the contrary, it is desirable that any majority consist of a mix of the constituent groups, and the federal structure helps to ensure that there is a mix and that the majority is distributed. 
In Australia the federation is on a geographical basis, and the aim of the federal structure is to ensure that a majority is geographically distributed, that it is not formed from the representatives of only a minority of states. It is not a violation of the federal principle that majorities are formed across state boundaries and that each state does not vote as a bloc. Such a situation is no more desirable than ethnic groups voting as blocs in the other kind of federation.
This constitutional theory was well understood by the framers of the Constitution, but it is not well understood now. It provides an example of the way in which political thought applied to political practice has declined in the last century. There is now an orthodox and facile treatment of federalism and the Senate, which is as follows. Because the constitutional framers used the expression "States' House" as a shorthand term to encapsulate the constitutional theory here expounded, it is assumed that they must have intended that all issues would be decided in the Senate on the basis of interests identifiable with particular states, and that senators from each state would vote as a bloc. Because this has not happened, it is said that the constitutional theory of the founders was obviously defective and the federal system and the Senate in particular have not worked. This pseudo-analysis is an insult to the intelligence of the founders, while revealing a great deal about the intelligence of those who present it. It has no basis in history, much less political theory. The principle of the double majority and the distributed majority does not rest on any such unrealistic assumptions.
A proper appreciation of the constitutional theory of the framers also indicates that the powers of the Senate are of greater importance than is often currently supposed. For the double majority to work a law must not be passed without the support of the required second majority, which means that the Senate must have equal powers with the House of Representatives in disposing of proposed laws. This demonstrates as fatuous any comparisons with the House of Lords and proposals to limit the Senate's legislative powers, particularly by removing its power over "supply", whatever that term is taken to mean by those who use it so readily.
An understanding of the underlying principle of the federal structure also gives greater significance to the double dissolution provisions in section 57 of the Constitution. These provisions were innovative for their time; when they were drafted no other comparable constitution had provisions for resolving deadlocks between the houses in a bicameral system. They are more than a mechanism for resolving deadlocks, however. They are a concession of federalism to democracy. Provided that the whole process set out in section 57 is followed, the double majority for the passage of laws may be dispensed with, only for the legislation causing the deadlock, and laws may be passed in accordance with the wishes of the majority of the representatives of the people as a whole, if that majority is not too narrow. In cases of significant disagreement, democratic representation prevails over the geographically distributed representation of the people. It must be remembered that laws have been passed in this way only once, in 1974, when there occurred the only double dissolution followed by a joint sitting of the Houses.
It is a commonplace observation that the founders sought to combine federalism with the British system of cabinet government, or so-called responsible government, whereby the composition of the lower house determines the composition of the executive and the executive must resign or go to an election if it loses the support of a majority of the lower house. The proponents of pure federalism at the constitutional conventions saw cabinet government as a significant departure from the federal system, but were outvoted in their attempts to jettison it. It is not such a great departure in theory, however. In the American system, the executive is elected by a body representing the people as a whole, as the electoral college gives the states representation in proportion to their population, with an additional geographical weighting. The practical significance of cabinet government for Australia is that it has developed into a system whereby the executive government controls one house of the legislature, and seeks to control the other by caucus discipline over its members. The interaction between the two Houses has thereby become an interaction between the executive government and the non-government majority in Senate.
The Senate in practice
Contrary to the orthodox non-analysis, the Senate has worked in practice in Australia in accordance with the theory on which it was founded. A double majority and a geographically distributed majority has been required for the passage of all laws. Except in the circumstance of laws passed at a joint sitting of the Houses after a double dissolution, laws have been passed only with the consent of a majority of the states speaking through their representatives. It has not been possible to form legislative majorities from a minority of states. To put it more simply, governments have not been able to rely for long solely on the support of Sydney and Melbourne while ignoring the rest of the country. This has avoided extreme alienation of the outlying parts of the country, in accordance with the main aim of federalism. The fact that the people of the states have voted for the same political parties has not removed this federalist underpinning of the Constitution, although, as has been indicated, the rigidity of the party system has weakened its effect.
When the party system produced unbalanced party majorities in the Senate, proportional representation was adopted for Senate elections. It is well known that proportional representation results in the party complexion of the Senate reflecting, more closely than that of the House of Representatives, the voting pattern of the electors. In spite of the great disparity in the sizes of the populations of the states, proportional representation awards seats in the Senate very nearly in proportion to shares of votes nationally.  While thereby producing what might be called an ideological distribution of the legislative majority, proportional representation, paradoxically, has also bolstered the Senate's function of requiring a geographically distributed majority. Because the party numbers are always so close in the Senate, the parties are further discouraged from ignoring the less populous states. Because every Senate seat is vital, every state is also vital.
The way in which federalism has worked in practice in Australia has been obscured not only by the orthodox treatment of the matter and the party system, but also by its nature as a safeguard. Safeguards often work without appearing to do so. An old lawyer and member of parliament from Alberta observed that he had practised law for 40 years without ever having applied for a writ of habeas corpus, but it would be rash to assume from this that the writ of habeas corpus is obsolete and should be abolished: it prevents arbitrary imprisonment because it is there. In the same way federalism has prevented by its existence government solely by Sydney and Melbourne.
The reference to Canada is apposite, because the example of Canada demonstrates, by the effects of the absence of a federal structure, the effects of having such a structure in Australia. Canada has several problems, some of which, such as the problem of Quebec nationalism, do not provide comparisons with Australia. One of those problems in recent times, however, has been the extreme alienation of the outlying provinces, particularly the western provinces, caused by the domination of government by the centres of population. So fed up did the western provinces become with the domination of the federal government by Toronto and Montreal (cf Sydney and Melbourne), that they spawned a new political party, the Reform Party, which was able virtually to wipe out one of the established major parties in a general election. While this may be seen as a fresh breeze blowing, such a geographical division bodes ill for the unity of the country. Such serious alienation has not occurred in Australia, and a primary reason for this is that the federal structure of the legislature, unlike the non-federal structure of the legislature in Canada, has altered the representational system by forcing majorities to be geographically distributed. It is significant that one of the demands of reformers in Canada is for a Senate like Australia's, representing the provinces equally and with real legislative powers. They refer to it as a "triple-E Senate", elected, equal and effective.  A disgruntled would-be politician from the western provinces told me that he favoured those provinces seceding from Canada and joining the United States. When asked why they would do such a thing, his first response was that they would each have two senators in Washington and therefore would not be ignored as they were ignored by Ottawa.
Political parties have helped to disguise the working of the federal system, but parties as such are not incompatible with that system. The founders were not so naive as to imagine that the electors of the states would not vote for parties. The problem is the rigidity of the party system and the factionalisation of parties. The founders did not envisage a situation whereby the leaders of the group which controls 51 per cent of the faction which controls 51 per cent of the parliamentary party which receives 40-odd per cent of the electorate's votes have absolute power to control the country. The significant point is that this party system not only weakens the federal structure but tends to break down parliamentary and representative government as such. Its effect on the House of Representatives and on so-called responsible government has been more devastating that its effect on federalism and the Senate. It has resulted in prime ministers who behave like emperors, even bullying speakers of the House of Representatives in public in sittings of the House, without people being aware that representative and parliamentary government as such has been repudiated.
The Senate, of course, performs parliamentary functions apart from its function of ensuring that legislative majorities are geographically distributed. It scrutinises proposed legislation and the activities of government, and inquires into matters of public concern. As governments make it their business to suppress these activities in lower houses, it can be said without much exaggeration that only the Senate performs these functions. It does not perform them as well as they could and should be performed, but there is always the hope of gradual improvement. It is the purpose of true parliamentary reform to foster that improvement. Much of what is called "reform", however, is designed to complete the stranglehold of the executive government over Parliament.
In recent times there has been a good deal of discussion about reforming the federal system, mainly generated by dissatisfaction with over-centralisation and by the need to devolve more real responsibility to the states. As part of that discussion, "reform" of the Senate is occasionally mentioned. It is perceived that, if the federal system needs reforming, so does one of its distinguishing features, the Senate.
Unfortunately, proposals for such "reform" usually rest on the historical and theoretical misconception which has been mentioned: it is thought that the Senate was intended to represent state governments and senators were to vote in state blocs on the instructions of their state governments, and the system should be changed to make this occur. Therefore there arise proposals to change to a German-style upper house, in which the members are the delegates of the state governments.  Such proposals dispense with the principle of senators representing the people of the states as distinct from the governments of the states. They also ignore the fact that state governments are controlled by particular political parties, and the voting of their delegates on national questions would reflect the party lines of those governments rather then their interpretation of the interests of their states. Such delegates might vote in state blocs, but they would also vote in party blocs, and we would be no nearer to achieving the unhistorical will-o'-the-wisp of a "States' House" as that term is misunderstood.
Proposals for an appointed Senate are also part of the fear of democracy which appears to be occurring in recent times. Wherever we look, there is a distrust of the electorate. The orthodox republicans do not want the electors to have anything to do with selecting a replacement head of state. So-called "peoples' conventions" are to have members appointed by governments, in stark contrast to the fully-elected convention (except for one state) of 1897-8. Suggestions for a return to an appointed upper house are of the same ilk. Such proposals demonstrate that the current political elite are far less democratic than their predecessors of the 1890s. It is sufficient to point out that the choice by the founders of an elected Senate representing the people of the states was an advance in constitutional construction, anticipating the 17th amendment of the United States constitution, and part of a trend to greater democracy in all aspects of government. To reverse that choice would be a backward step.
In any event, as the foregoing has suggested, the real need for reform is not so much in the institutions of government as in the political parties. They have become narrowly based, factionalised, undemocratic oligarchies, apt to be controlled by too few people, closed to public view but open to manipulation and outright corruption. Reforming them would make the institutions of government work better without changing those institutions, but without reforming them the institutions cannot work very much better than they do at present.