Welcome to
Science a GoGo's
Discussion Forums
Please keep your postings on-topic or they will be moved to a galaxy far, far away.
Your use of this forum indicates your agreement to our terms of use.
So that we remain spam-free, please note that all posts by new users are moderated.


The Forums
General Science Talk        Not-Quite-Science        Climate Change Discussion        Physics Forum        Science Fiction

Who's Online
0 registered (), 226 Guests and 1 Spider online.
Key: Admin, Global Mod, Mod
Latest Posts
Is there anybody out there?
by paul
12/07/19 03:58 AM
Top Posters (30 Days)
True 1
paul 1
Page 2 of 7 < 1 2 3 4 5 6 7 >
Topic Options
#16893 - 10/31/06 05:47 PM Re: Australia hasn't even warmed up: stats prove it
Count Iblis II Offline
Senior Member

Registered: 01/21/05
Posts: 375
Richard, I have to agree with Daniel here. I'm not saying that there isn't scientific fraud and that the blogger is wrong. But until the results are retracted by the scientific journals in which the results have been published, we cannot just assume that some blogger posting comments is correct.

There are so many bloggers with so many agendas that you can almost always find a blogger who is refuting a given scientific result.

I also don't think that you can very easily use the freely available data to check the climate change claims. You have to a lot of non trivial data analysis to see the few tenths of a degree increase in average tempperatures.

To be credible you have to do a Monte Carlo simulation of your data analysis method where you replace the actual data with simulated data taken from a model according to which

a) there is no trend

b) there is trend toward higher temperatures.

etc.

Top
.
#16894 - 10/31/06 05:47 PM Re: Australia hasn't even warmed up: stats prove it
DA Morgan Offline
Megastar

Registered: 10/17/04
Posts: 4136
Loc: Seattle, WA
RicS wrote:
"Once again the climate thread has devolved into an argument about individual's and their relative "worth""

No it hasn't. It has focused on the fact that the premier research organization in Australia is a far more credible source of information than a single blogger promoting his personal website.

If you are going to claim CSIRO is wrong you need to produce data ... not volume of arguments. Brevit with a link to real data trumps 594 words of argument.
_________________________
DA Morgan

Top
#16895 - 10/31/06 05:53 PM Re: Australia hasn't even warmed up: stats prove it
DA Morgan Offline
Megastar

Registered: 10/17/04
Posts: 4136
Loc: Seattle, WA
Jonathan Lowe wrote:
"However, data does not lie"

Data doesn't tell the truth either. Data is just points on a line.

What counts is interpretation and which statistical methods are applied to it.

As Samuel Clemens (Mark Twain) said so profoundly:
"There are three types of liars: Liars, damned Liars and statisticians."

Do I trust your analysis over that of PhD climatologists worldwide including those at CSIRO? Buy yourself a stubbie because you already know the answer.

My basic take on what your wrote is that you have suspended the rules of physics. That warmer water has no affect on climate. That warmer water has no affect on storms. That the climate of the rest of the planet doesn't affect that in Oz. And I'm not willing to get drunk enough to agree anytime soon.
_________________________
DA Morgan

Top
#16896 - 10/31/06 07:11 PM Re: Australia hasn't even warmed up: stats prove it
dehammer Offline
Megastar

Registered: 03/23/06
Posts: 1089
lets see, do i trust my own eyes or do i trust a politically motivated, highly opinionated group working with preconceived set of conclusions, using selected parts of the data to prove that conclusion? hummm, tough decision there.
_________________________
the more man learns, the more he realises, he really does not know anything.

Top
#16897 - 10/31/06 07:15 PM Re: Australia hasn't even warmed up: stats prove it
dehammer Offline
Megastar

Registered: 03/23/06
Posts: 1089
Quote:
Originally posted by Count Iblis II:
Richard, I have to agree with Daniel here. I'm not saying that there isn't scientific fraud and that the blogger is wrong. But until the results are retracted by the scientific journals in which the results have been published, we cannot just assume that some blogger posting comments is correct.
so you will not believe it untill a group of egotisital eggheads paid to prove a point admit that they "adjusted" the data to prove it AND they get a journal that is paid to pass on threatening information because that is the type that sells subscription agrees to publish the info that there is no danger. Untill these happen, you will not trust your own eyes to look at the data itself?
_________________________
the more man learns, the more he realises, he really does not know anything.

Top
#16898 - 10/31/06 09:05 PM Re: Australia hasn't even warmed up: stats prove it
DA Morgan Offline
Megastar

Registered: 10/17/04
Posts: 4136
Loc: Seattle, WA
dehammer wrote:
"lets see, do i trust my own eyes"

In science the first rule is not too. Apparently you haven't learned that lesson.

For example ... your eyes will clearly tell you that the moon orbits the earth ... yet any astrophysicist will tell you that it doesn't.
_________________________
DA Morgan

Top
#16899 - 10/31/06 09:42 PM Re: Australia hasn't even warmed up: stats prove it
Count Iblis II Offline
Senior Member

Registered: 01/21/05
Posts: 375
Quote:
Originally posted by dehammer:
Quote:
Originally posted by Count Iblis II:
Richard, I have to agree with Daniel here. I'm not saying that there isn't scientific fraud and that the blogger is wrong. But until the results are retracted by the scientific journals in which the results have been published, we cannot just assume that some blogger posting comments is correct.
so you will not believe it untill a group of egotisital eggheads paid to prove a point admit that they "adjusted" the data to prove it AND they get a journal that is paid to pass on threatening information because that is the type that sells subscription agrees to publish the info that there is no danger. Untill these happen, you will not trust your own eyes to look at the data itself?
Dehammer, if you really believe what you wrote then can prove your point by following Alan Sokal\'s example. Just emulate what you think climate scientists are doing. So, take some data and doctor/manipulate it so that it confirms global warming (instead of doing bona fide data analysis). Write up an article and send it for peer review to a leading journal. You can put below your name that you work for NASA or some other institution, they usually don't do background checks to verify your affiliation.

If your article is accepted for publication you will have proven that it is easy to publish bogus research in peer reviewed journal. It still won't prove that most of the published research is bogus...

Top
#16900 - 10/31/06 10:19 PM Re: Australia hasn't even warmed up: stats prove it
JonathanLowe Offline
Member

Registered: 10/29/06
Posts: 87
Loc: Melbourne
Hmm no idea how monte carlo simulations has anything at all to do with temperature analysis, but anyway.

I can admit that some people will obviously respect Climate scientists who are working int he field over me, just some blogger (a blogger who has spent 9 years at university studying statistcs by the way), and I can understand that people are skeptical of what I have found, however....

I will be publishing on my website how exactly I came to the conclusions and exactly what methods I used, so that, should you wish, you can also get the data from the ABM and replicate exactly what I've done in the way that I suggested and prove to yourself, that my analysis is unbiased and accurate.

Top
#16901 - 10/31/06 11:19 PM Re: Australia hasn't even warmed up: stats prove it
RicS Offline
Senior Member

Registered: 03/26/06
Posts: 310
Loc: Sydney, Australia
G'day Count,

I didn't suggest taking a "blogger" on face value. I did suggest that the data analysis shown by Mr Lowe has some merit, unless someone is able to dispute what he has done.

The analysis of cyclones is nothing new at all and conforms to published research. James Cook University has done similar research and come to similar conclusions.

The analysis of drought is a little different and Mr Lowe has chosen to use selective data in some examples he uses. This is where I would suggest anyone who disagrees with the analysis would be able to have a basis to argue. But only a basis. It would start a discussion that it would seem that Mr Lowe is quite prepared to continue on this forum. I, for one, would welcome that.

However, water, being very important in Australia, has been subject to extensive analysis and Mr Lowe's analysis of drought is also not particularly unique or earth shattering. I pointed out in another thread that the threat of global warming has caused productive farmland, currently experiencing five years or so of far below average rainfall, to drop in value by 30% in the last three or so months. The farmers, and the realtors agreed that the drop was because of the fear that this was not an "ordinary" drought but rather a taste of what is to come because of global warming.

Actually, global warming, does not necessarily cause the productive Eastern Australian farmlands to become more drought prone. But the fear has caused real people to suffer terrible losses. These are farms that are amongst the most efficient in the world. Australian farmers do get drought relief but they do not get the myriad of subsidies enjoyed by European farmers or even the special protections, subsidies and other advantages that US farmers get.

The three big exporters of grain in the world are Australia, Canada and the US. The fear of global warming damaging Australia's capacity to produce and export grain would have a very marked effect on the world.

And Count, the graphs and data for cyclones, for instance, does not require very much analysis at all. The figures are not particularly complex and Mr Lowe's graphs are well set out. The drought data is more complex and I would agree with you that a lay person would not be able to look at what he has done and spot flaws quickly if they were there. In that respect, I was thinking more of myself and others that do look at climate data as a normal part of their day.

But feel free to comment on any of the data in respect to drought or even raise questions and, if Mr Lowe, is not available at the time to respond, I will be happy to do so.

As to scientific publications and research relating to data, this is one area where research is actually difficult to come across. Much of it pre-dates the Internet. The research is often complex in its terminology and even in its assumptions.

I have a thread that I'm going to post soon looking at a research paper that attempts to prove that urban effect is inconsequential. I believe I can present it here so that even those that have no background in data analysis will be able to understand the principals and what I consider serious flaws.

As to data research being reviewed in other published research, it just does not happen in climate science. So Mr Lowe's blog information may be the best you are likely to get.

The real classic example of misuse of data was the research of Ms Oreskes in relation to the "consensus" of climate research. This research was blatantly biased, in my personal opinion. It was an example of just what not to do with data analysis. A number of scientists, often in the field of data analysis, or in fields relating to scientific methods, have produced research papers demonstrating just how flawed this research is. The publisher would not even publish a letter critical of the research, let alone publish any research papers showing quite different results using the same source of data.

This is also true for urban effect and the analysis of Surface Air Data data sets. They just do not get published.


Regards


Richard
_________________________
Sane=fits in. Unreasonable=world needs to fit to him. All Progress requires unreasonableness

Top
#16902 - 11/01/06 01:33 AM Re: Australia hasn't even warmed up: stats prove it
DA Morgan Offline
Megastar

Registered: 10/17/04
Posts: 4136
Loc: Seattle, WA
Jonathan wrote:
"Hmm no idea how monte carlo simulations has anything at all to do with temperature analysis"

They do and they are important. Google for:
"Climate" and "Monte Carlo Simulation"

The first link I find is:
http://physics.gac.edu/~huber/envision/instruct/montecar.htm

which contains the following:
Now, for the problem we are studying, namely determining the global climate, there are several places where a Monte Carlo simulation can be of use. In particular, we will need it to help us determine the global average temperature and the amount of sunlight which falls into each latitude band. To determine the global average temperature, we want to average of the temperature of each latitude band, but there is obviously much more land area in the region from the equator to a latitude of 10o than in a band from 80o to the north pole. Therefore to determine the average temperature, we will want to weight the temperature of each band by the fraction of the earths land area in that band. We can do this analytically using integration, but this also can be done well with a Monte Carlo method.

We will modify the program above to generate random (XYZ) points in a cube of sides 1 unit. Next we will determine if they are on the surface of a sphere of radius 1 by using the following:

Rrand = Xrand.^2 + Yrand.^2 + Zrand.^2;
CheckValue = Rrand<=1.01 & Rrand>=.99;

this will determine if the points are on the surface of the sphere. Next, we will check if points are within each latitude band as well as on the surface of the sphere. The program will increment a counter for each point which meets these criteria. At the end, we can divide the number in each latitude band by the total number of points which were on the surface to find the area in each band.

Like I said above. It is not the data I don't trust ... it is the statistical methodology and the conclusions.
_________________________
DA Morgan

Top
#16903 - 11/01/06 01:58 AM Re: Australia hasn't even warmed up: stats prove it
JonathanLowe Offline
Member

Registered: 10/29/06
Posts: 87
Loc: Melbourne
Great response Rics. The depth of conversation much appreciated. The great analytical debate on this forum far outweighs any other I have seen.
DA Morgan, being a statistician and studying it at university for 8-9 years, I do know a lot about Monte Carlo analysis. In fact I started my Masters Thesis on it, before changing. Obviously, it's basically a simulation. But if you have the data, there's no reason to simulate it if you are looking at trend analysis.

He's basically using a simulation to get random areas of which to grab temperatures from, something which isn't necessary if just studying Australia. If he used longtitudes as well instead of just latitudes, then monte carlo would also be not necessary

Top
#16904 - 11/01/06 03:43 AM Re: Australia hasn't even warmed up: stats prove it
RicS Offline
Senior Member

Registered: 03/26/06
Posts: 310
Loc: Sydney, Australia
G'day Dan,

Something that is science. Thank you. That isn't sarcasm by the way. I do thank you for actually going to the trouble to look at the science.

The Monte Carlo method is of use if you are trying to determine a "true" average. But it is of no relevance at all if all you wish to do is look at anomolies over time. A regional analysis will do just fine for that and even ignoring concentrations will still provide a basis for comparison. The more you "adjust" the data for various deficiencis in it, the more prospects your have of manipulating the data.

So if I was faced with the choice of having 5,000 weather stations that had long term daily averages and weighing them to take into account their latitude or to simply compare them year to year, my preference would be the simplest one. Actually I'd do both but the Monte Carlo method does not account for the distribution of oceans and their effect on temperatures or a considerable number of other variables that affect the usefulness of a particular weather station as respresentative of its region. A weather station at a say 30 degrees north on the coast is going to be quite different to one 1,000 kilometres inland. The affect of the moderating influence of the ocean is much greater for most latitudes than the position latitudinally, assuming we are talking comparisons over time.

Having said all that, this has very little to do with Australia. The distribution of weather stations is markedly coastal but in determining whether the "average" temperature has changed over time, it really matters not much at all whether you simply use all available weather stations or you give weight to such things as concentrations of stations, distance from oceans, height etc.

Actually the distribution of weather stations isn't all that bad in Australia if those stations with considerable urban effect are taken out. You still have the centre under-represented but there are stations throughout Australia.

Mr Lowe should be able to demonstrate the various ways of determining an average for Australia far better than I. His expertise is in data analysis, something I have only had to do as part of an overall analysis of the scientific methodologies adopted. I will be greatly interested to see Mr Lowe's take on Australi's temperatures. Actually I'd welcome comments on any mistakes or poor assumptions I've made in this post.

We are at least back to talking about the science of climate and that is a terrific thing.


Regards


Richard
_________________________
Sane=fits in. Unreasonable=world needs to fit to him. All Progress requires unreasonableness

Top
#16905 - 11/01/06 01:23 PM Re: Australia hasn't even warmed up: stats prove it
Count Iblis II Offline
Senior Member

Registered: 01/21/05
Posts: 375
Quote:
Originally posted by JonathanLowe:
Hmm no idea how monte carlo simulations has anything at all to do with temperature analysis, but anyway.

I can admit that some people will obviously respect Climate scientists who are working int he field over me, just some blogger (a blogger who has spent 9 years at university studying statistcs by the way), and I can understand that people are skeptical of what I have found, however....

I will be publishing on my website how exactly I came to the conclusions and exactly what methods I used, so that, should you wish, you can also get the data from the ABM and replicate exactly what I've done in the way that I suggested and prove to yourself, that my analysis is unbiased and accurate.
Besides Dan's example, you need to prove that your method works. If people use methods for data analysis that are not standard or perhaps they are standard but applied in a slightly diferent way than usual, then you need to validate your method. There can be small subtle effects that can affect the outcome of the analysis. So, you need to test it just like you would test software to debug it: by doing experiments where you input data for which the outcome is known.

So, you treat your data analysis method as a black box. Data comes in and results come out. You then simulate the data that you would expect from weather stations for different climate change scenarios. This is where the Monte Carlo method comes in. The fake data is the trend plus local variability which is random on various time scales.

Using the simulations you can see how good your method is. E.g. at what rate must average temperatures increase for your method to detect the increase with 95% probability?

Top
#16906 - 11/01/06 01:31 PM Re: Australia hasn't even warmed up: stats prove it
Count Iblis II Offline
Senior Member

Registered: 01/21/05
Posts: 375
Richard,

I'm a bit skeptical at the claim that critical articles (that are not flawed) don't get published. The Oreskes thing is just one controversial example. If there is a problem then there should be many more people besides Peiser who are complaining.

Top
#16907 - 11/01/06 02:40 PM Re: Australia hasn't even warmed up: stats prove it
JonathanLowe Offline
Member

Registered: 10/29/06
Posts: 87
Loc: Melbourne
Count Iblis II, can I ask you what your credentials are in Statistical analysis? No offence, but doing a monte carlo analysis based on your assumptions of what you think the weather stations should say based on climate change and then making 95% confidence intervals based on these simulations I believe is absurd.

Better than that. Why not get the data, test the data for trend analysis. Who needs monte carlo simulations for data that is already there, of which should you do them is subject to what you think should happen in the first place. This is not good statistical analysis.

Top
#16908 - 11/01/06 04:09 PM Re: Australia hasn't even warmed up: stats prove it
Count Iblis II Offline
Senior Member

Registered: 01/21/05
Posts: 375
Quote:
Originally posted by JonathanLowe:
Count Iblis II, can I ask you what your credentials are in Statistical analysis? No offence, but doing a monte carlo analysis based on your assumptions of what you think the weather stations should say based on climate change and then making 95% confidence intervals based on these simulations I believe is absurd.

Better than that. Why not get the data, test the data for trend analysis. Who needs monte carlo simulations for data that is already there, of which should you do them is subject to what you think should happen in the first place. This is not good statistical analysis.
Absurd? I don't think you get it. How can you conclude anything based on your measurements? What are the error bars. What Global Warming scenarios are constrained given your conclusions?


The Monte Carlo method is necessary to validate your method. If you don't do it, no one will take you serious. It is standard practice in many scientific fields ranging from astrophysics, particle physics etc. The people at CERN who will soon be swamped with thousands of terabytes of data are busy doing simulations using the software that they'll later use to extract properties of particles they hope to discover when the Large Hadron Collider becomes operational. They need to be sure that if they see a Higgs particle, it is indeed a Higgs particle and not some software bug, or perhaps some sublte flaw in the reasoning that led to a flawed algorithm used in the data analyses.

They run simulations of the detectors themselves to generate the data that would be produced according to certain theoretical scenarios. They then run their data analysis software on the (fake) data to see if what comes out using is indeed consistent with what they put in.


If you only rely on "theory" to validate your data analysis technique you are unlike to spot any subtle flaws. In your case, no one will have any confidence that, if there is a trend toward higher average temperatures consistent with what most climate scientists think is going on, you would able to pick it up using your method.

Average global temperatures have only increased by 0.6 degrees C over a period of more than 100 years. That's a small increase over a large period and will be completely swamped by natural fluctuations at indivudual stations. The signal only becomes visible after averaging over a large number of stations.

Top
#16909 - 11/01/06 05:08 PM Re: Australia hasn't even warmed up: stats prove it
dehammer Offline
Megastar

Registered: 03/23/06
Posts: 1089
Quote:
Originally posted by DA Morgan:
dehammer wrote:
"lets see, do i trust my own eyes"

In science the first rule is not too. Apparently you haven't learned that lesson.

For example ... your eyes will clearly tell you that the moon orbits the earth ... yet any astrophysicist will tell you that it doesn't.
so the earth orbits the moon?

the amount you can trust your eyes to tell the truth is dependent on the amount of actual data that you have.

technically, on a simplistic level, the moon does orbit the earth. its only when you get more specific that you get to the level that its both the earth and moon orbiting a center point. the fact that the point is underneith the earths crust just means that on an elementary level its sufficent to say the moon orbits the earth.

if you have suffient data then you can accept what the eyes are seeing.

if the data says that at 90 percent of the stations the level of the sea has not changed, and only in two stations, that happen to be on top of old river deltas, has the land to sea level change, then its sufficent to go by what the facts are in front of your eyes, rather than going by a political party that ignores the 90 percent and only uses the river delta station.

there have been several studies around the world, used to determine how much the land beneith cities are sinking compared to surrounding lands, that have had the data from the land that is sinking used to prove the sea is rising.

Louisiana is a good example. any geologist will tell you the land is sinking as the water from the mud deposited there by the river when it was a delta is forced out by the weight of the soil above it. yet where do global warming alarmist go to for proof that the sea is rising? Louisiana, of course.
_________________________
the more man learns, the more he realises, he really does not know anything.

Top
#16910 - 11/01/06 05:35 PM Re: Australia hasn't even warmed up: stats prove it
DA Morgan Offline
Megastar

Registered: 10/17/04
Posts: 4136
Loc: Seattle, WA
Ric ... I never said a Monte Carlo simulation was important or required. What I said was it is the analysis and conclusions that matter ... not just the data points.

That Jonathan was unaware of the "possible" value of this methodology puts into question what method was used and the validity of the conclusions.

Again: I'll trust the laws of physics first. CSIRO, NOAA, and NASA second. Some blogger, no matter how well intended or insightful, in about 90 years when history shows he was correct. The scientific method does not include blogging and personal opinion.
_________________________
DA Morgan

Top
#16911 - 11/01/06 10:19 PM Re: Australia hasn't even warmed up: stats prove it
JonathanLowe Offline
Member

Registered: 10/29/06
Posts: 87
Loc: Melbourne
ok a few things:
DA Morgan - I do know about Monte Carlo Simulation. I actually use Monte Carlo Simulations in my work all the time, almost every day. I studied them in one of my 8-9 years at university studying statistics and the analysis of data. I am aware of its benefits. And I am also aware of when it should be used. One time is when predicting for example. Which of course I am not trying to do at all.

Whilst Count Iblis II, did not answer my question about his statistical credentials, monte carlo is not the best method to determine trend analysis. Time series analysis, what is was made for, is simply better. Monte Carlo can be used to create error bars (side note - error bars are something that a lot of non-statisticians use for some reason), but the practicle use of it is limited unless you want to predict.

Monte Carlo is based on the data that you currently have, and hence any simulation is a random representation of the data you have at current. Hence an analysis of the data that you curently have is what we are after. You can use Monte Carlo methods to predict, based on the data that you have, what will happened in the future. I'm all good with that, and it's something that I might do. But at the moment, I am just analysing the data that I currently have.

Top
#16912 - 11/01/06 10:57 PM Re: Australia hasn't even warmed up: stats prove it
DA Morgan Offline
Megastar

Registered: 10/17/04
Posts: 4136
Loc: Seattle, WA
Excuse me Jonathan but if you go to the previous page you will find that you wrote:
"Hmm no idea how monte carlo simulations has anything at all to do with temperature analysis...."

Now you write:
"I actually use Monte Carlo Simulations in my work all the time, almost every day."

Perhaps you can reconcile these two statements but I can not. First you don't know what they have to do with the topic and then you use them every day. It seems to me you have shot yourself in the foot.
_________________________
DA Morgan

Top
Page 2 of 7 < 1 2 3 4 5 6 7 >



Newest Members
debbieevans, bkhj, jackk, Johnmattison, RacerGT
865 Registered Users
Sponsor
Facebook

We're on Facebook
Join Our Group

Science a GoGo's Home Page | Terms of Use | Privacy Policy | Contact Us
Features | News | Books | Physics | Space | Climate Change | Health | Technology | Natural World

Copyright © 1998 - 2016 Science a GoGo and its licensors. All rights reserved.