Checking AI answers to 30 first home buyer money questions

Counts, not rates. 30 questions, 2 systems, 61 answer slots including one sensitivity check, 59 answers scored (two Google searches returned no AI Overview). One capture day. The two systems were reached in different ways and asked in different forms, so this page does not rank them. Two of the 60 slots in the agreement count had no AI Overview; on the 58 answers that were scored, the scorers agreed on 57. Both scorers were AI models from one family, and the ChatGPT answers came through a third party capture service; the limits section explains both. Nothing here estimates how often AI is right or wrong on other questions, other days or other products.

What we asked

Sensitivity check, Google only: Q03S, the Queensland grant searched as “first home buyers grant qld”. It is reported on its own line and never added to the 30.

Four questions use older scheme names because people still search them: First Home Guarantee (Q18, Q20 and Q22) and Home Guarantee Scheme (Q21). The official answer for each is taken from the current 5% Deposit Scheme page. The Northern Territory house and land question is not limited to first home buyers. Two questions (Q12 and Q16) carry a timing phrase in the ChatGPT prompt that the shorter Google query does not; the substance is the same.

Results by question

Sensitivity check

• Scorable means an AI Overview was shown. Two Google searches returned none and are counted separately.
• Out of date means the answer matched a rule the official record shows has since changed; the change date is shown. Wrong means incorrect for the question as asked with no earlier official rule we could verify. Incomplete means no single checkable current figure was given.
• Differences are official figure minus stated figure, written as understatement or overstatement. They are shown separately and never added together. None is a measured cost to any buyer.
• (note) marks a correct answer that carries a note explained below the table.
• The Northern Territory house and land question (Q17) is not limited to first home buyers.

Results by topic family

The sensitivity check (Q03S, Google only) was out of date. It is reported here on its own line and is not in the 30.

The five Google AI Overviews that were not correct

Queensland first home owner grant (Q03, Google AI Overview: out of date)

The question asked how much the grant is. The Queensland Government and Queensland Revenue Office pages say $30,000 for eligible contracts signed on or after 20 November 2023, continuing for eligible contracts signed from 1 July 2026, for a new home valued at less than $750,000 including land. The AI Overview gave the grant as $15,000 and presented the $30,000 rate as a window that ended on 30 June 2026. The $15,000 figure matches the rule that applied to contracts signed before 20 November 2023, and the 30 June 2026 end date matches the deadline the Queensland Revenue Office published in its June 2025 budget note before the June 2026 budget note continued the grant. Both are superseded official rules, so the class is out of date rather than wrong. Grant amount difference: $15,000 understatement. The same result appeared on the sensitivity check search that used the words “buyers grant”. Official source: the Queensland Government first home owner grant page.

Northern Territory first home owner grant (Q08, Google AI Overview: out of date)

The question asked how much the grant is. The nt.gov.au page says the HomeGrown Territory Grant is $50,000 for first home buyers buying or building a new home, for contracts signed between 1 October 2024 and 30 September 2027, with no price cap. The AI Overview presented the $10,000 grant for an established home as still available. That grant applied only to contracts signed between 1 October 2024 and 30 September 2025, so the class is out of date. There is no dollar difference to compute: a closed grant presented as open is an eligibility claim, which we count but do not convert to dollars. Official source: the nt.gov.au HomeGrown Territory Grant page.

Northern Territory stamp duty on a house and land package (Q17, Google AI Overview: wrong)

The question asked whether you pay stamp duty on a house and land package in the Northern Territory if you sign the contract now. The nt.gov.au page says a house and land package bought from a building contractor in a single transaction, with a contract signed between 1 July 2022 and 30 June 2027, is exempt from stamp duty; the exemption is not means tested and has no cap on the value of the property. The answer said stamp duty is payable on the land part of the package and did not mention the house and land package exemption. We found no verified earlier official rule that matches it, so under our rubric it is classed wrong. The Territory Revenue Office guideline on home owner assistance (version 16, in effect from 1 July 2022) was read on 2 November 2026 through a stored copy and on 4 November 2026 in a browser; it sets out the exemption from 1 July 2022 and says nothing about how duty applied to a house and land package before that date, so the earlier history is not verified, and we flag that. No dollar figure is computed because the question stated no property value. This question is not limited to first home buyers. Official source: the nt.gov.au stamp duty exemption page.

Help to Buy income limits (Q23, Google AI Overview: incomplete)

The question asked what the income limit is for Help to Buy in Australia. The firsthomebuyers.gov.au scheme page and its threshold table give $103,000 for an individual applicant and $165,000 for single parents and joint applicants, assessed on the notice of assessment for the previous financial year, for the period 1 July 2026 to 30 June 2027. The AI Overview gave each limit as a range running from the earlier figures to the current ones, without saying which applies now. Under our rubric a range that contains the official figure, with no single current figure, means the required claim was not stated, so the class is incomplete, not wrong. No difference is computed from a range. Our guide: Help to Buy.

Help to Buy price cap in Sydney (Q24, Google AI Overview: out of date)

The question asked the Help to Buy price cap in Sydney. The firsthomebuyers.gov.au price cap table gives $1,300,000 for Sydney and New South Wales regional centres and $800,000 for other areas of the state. The AI Overview gave the Sydney cap correctly. It also volunteered the scheme’s income limits and presented the limits for 1 July 2025 to 30 June 2026 ($100,000 and $160,000) as current. Those limits are the verified earlier rule recorded against the income limit question (Q23); they were indexed to $103,000 and $165,000 from 1 July 2026. Because a buyer earning between the old and new limits would be told they do not qualify, the owner decided this volunteered claim is material, and the class is out of date. That was a judgement, and the reason is given so a reader can disagree. Threshold differences: $3,000 understatement for an individual applicant and $5,000 understatement for single parents and joint applicants, shown separately. Our guide: Help to Buy.

Of these five, three were out of date: each matched a rule that changed between 1 October 2025 and 1 July 2026, and one of them also used a grant amount superseded on 20 November 2023. One was incomplete. One matched no earlier official rule we could verify.

Three correct answers that carry a note

Three answers were scored correct and carry a note.

Google, Q07, the ACT grant. The answer was correct on the question asked: the ACT has had no first home owner grant for transactions from 1 July 2019, when the Home Buyer Concession Scheme replaced it. The answer also volunteered income test figures and price caps for that replacement scheme, presented as current. They match no official rule we could verify, and for transactions from 1 July 2026 the scheme has no income threshold and no value limit. Because the question was about the grant, the class is unchanged. We record the error here so the count of correct answers does not hide it. We do not print the figures the answer gave.

Google, Q16, ACT stamp duty. The answer was correct on every claim the question required. It also made a statement about which of the buyers must live in the home. Our official evidence for this question does not cover that rule, so the statement was not scored and the answer carries an uncertainty note.

ChatGPT, Q25, the Help to Buy share. The answer gave the government contribution shares correctly. It also volunteered the current income limits with an earlier financial year label. One scorer classed the label an error and the other did not. This was the one disagreement between the scorers; the agreement section explains how it was decided.

Where the two AI Overviews were missing

For two of the 30 Google searches the request succeeded and no AI Overview was returned: the Western Australian grant (Q04) and the First Home Super Saver Scheme total limit (Q26). We report these as absent, not as wrong, and they are left out of the 28 scorable Google answers. Five Google requests failed on the first attempt with an error reported through the capture service (status 40101); all five succeeded when retried ten minutes later, so no slot is reported as a technical failure.

How often the two scorers agreed, and who decided

Every answer was scored twice, by two different AI models working independently from the same saved captures and official pages with the published rubric; the two sets of scores were compared before any disagreement was resolved. Before adjudication the scorers agreed on the answer class in 59 of 60 answer slots (30 questions, 2 systems). Two of those slots had no AI Overview, so on the 58 answers that were scored the scorers agreed on 57. They also agreed on the sensitivity check. On the 3 answers that carried a figure to extract (Q03, Q03S and Q24 on Google), both scorers read the same figures. The one disagreement was the ChatGPT answer to Q25 (how much the government contributes under Help to Buy): the answer gave the correct contribution shares and also volunteered the current income limits with an earlier financial year label. One scorer called the label an error, the other did not.

The owner of NestPath decided the one disagreement and six further scoring questions raised in the adjudication list; in all seven cases the decision accepted the recommendation drafted by scorer 1, and the reasons are published. A separate AI review seat checked the recommendations against the official sources. The disagreement is counted in the agreement rate above. The seven decisions were:

1. ChatGPT Q25: correct, with a label note. The income limit figures are the current ones for 1 July 2026 to 30 June 2027, and the official page assesses them on the notice of assessment for the 2026 financial year, so the earlier year label is defensible.
2. Google Q03 and Q03S: out of date, not wrong. The $15,000 figure matches the verified rule that applied before 20 November 2023, and the Queensland Government page says the $30,000 grant continues for contracts signed from 1 July 2026.
3. Google Q07: correct. The volunteered figures concern a different scheme from the grant asked about; the error is noted above.
4. Google Q17: wrong. The answer said duty is payable, which answers the question incorrectly; the House and Land Package Exemption gives no duty for a qualifying package.
5. Google Q24: out of date. A superseded income limit for the same scheme, presented as current, is material because a buyer earning between the old and new limits would be told they do not qualify.
6. Google Q23: incomplete, under the range rule.
7. Google Q16: correct, with an uncertainty note on the residence statement.

Did the AI cite NestPath?

We record for every answer whether it cited any NestPath page. One did: the Google AI Overview for the New South Wales grant (Q01) cited our New South Wales grants page, and the capture log shows that page ninth in the ordinary results for that search. That answer was scored correct by both scorers. No other answer, on Google or ChatGPT, cited NestPath.

How we did it

What we checked

We wrote 30 short questions of the kind Australian first home buyers type into a search box: the first home owner grant in every state and territory, first home buyer stamp duty rules in every state and territory, the Australian Government 5% Deposit Scheme price caps and income rule, Help to Buy income limits, price caps and the government share, the First Home Super Saver Scheme contribution limits, two lenders mortgage insurance basics, and the Reserve Bank cash rate. The questions were fixed on 2 November 2026 before anything was captured, and they are published in full above. Four questions use older scheme names (First Home Guarantee, Home Guarantee Scheme) because people still search those names. One question (Northern Territory stamp duty) is about house and land packages and is not limited to first home buyers. The questions are ours; they are not a random sample of what people ask, several overlap, and the results describe these 30 questions on one day, not the share of all first home buyer questions that AI gets right or wrong. A thirty first search, the Queensland grant with the words “buyers grant” in place of “owners grant”, was decided before capture as a sensitivity check and is reported on its own line, never added to the 30.

The official answers

Each question has one or more official sources: a state or territory revenue office, a state government grants page, the Australian Government first home buyers site, the Australian Taxation Office, Moneysmart or the Reserve Bank of Australia. Published rules were checked on 2 November 2026 and again on 3 November 2026, in the same minutes as the captures. We did not use comparison sites, news reports or our own pages as sources. For every source we record the date shown on the page, the machine readable date in the page where there is one, and the time we saved it; where a page shows no date we say so rather than invent one. Two official sites block automated requests, so their pages were saved as page text from an ordinary browser on 2 November; on 3 November four of those seven pages were read again in the browser, and for the other three the 2 November text is the evidence. The saved pages and their checksums are listed below. On 3 November, 26 of 33 page fetches returned the page and 7 were blocked; 17 of the 26 had a different checksum from 2 November. For 16 of the 17 the difference was page furniture such as menus and tokens. The Reserve Bank cash rate table had gained the 4 November 2026 row (4.85 per cent) after the 3 November decision, and that is the figure the cash rate question was scored against. No other official answer changed between the two checks. Where an official page states an eligibility period, we report that period as the page states it.

What we captured and when

Google. On 3 November 2026, between 14:11 and 14:24 UTC (between 01:11 and 01:24 on 4 November in Sydney and Melbourne, which were on daylight saving time), we requested the Google search results page for each question through the DataForSEO SERP API, with the location set to Australia (DataForSEO location code 2036), language English, device desktop, and the option that loads AI Overviews that Google adds after the page loads switched on. We saved the full response for every query. An AI Overview was counted as present when the response contained one; it was counted as absent when the request succeeded and no AI Overview was returned; a failed request is reported as a failed request, not as an absent AI Overview. Five of the 31 Google requests failed on the first attempt with an error reported through the capture service (status 40101), for questions Q14, Q18, Q22, Q25 and Q27, between 14:11 and 14:12 UTC. Each was retried once, ten minutes later, between 14:21 and 14:22 UTC, and all five succeeded. Final count: 29 of 31 searches showed an AI Overview, 2 did not (Q04 and Q26), 0 failed.

ChatGPT. On 3 November 2026, between 14:12 and 14:24 UTC, we put each question, word for word with nothing added, to ChatGPT through the DataForSEO ChatGPT capture service, which submits the question and returns the answer text, the sources it showed, the model name it reported and a timestamp. We did not force web search. This is not the same as a person typing into chatgpt.com, and the model is whichever one ChatGPT used on the day. We did not test whether another time, place or route would give a different answer. The model name the service returned was the same for all 30 answers: “gpt-6”. All 30 requests succeeded on the first attempt.

Reserve Bank. The cash rate question was captured on both systems at 14:23 UTC on 3 November 2026 (01:23 on 4 November in Sydney), after the Reserve Bank had published its decision of 3 November 2026, and the Reserve Bank cash rate page was saved at 14:23 UTC in the same minute. The Reserve Bank table showed a cash rate target of 4.85 per cent with an effective date of 4 November 2026, and that is the figure the answers were scored against. All other questions were captured between 14:11 and 14:23 UTC.

How we scored

Each answer was broken into claims (the amount, the threshold, who qualifies, the dates, which scheme). Each claim was scored against the official page as correct, out of date (the claim matches a rule that the official record shows has since changed, and we publish the change date), or wrong (incorrect for the question as asked, with no earlier official rule we could verify). An answer is correct when every claim it needed to make is correct; out of date when at least one claim is out of date and none is wrong; wrong when any claim is wrong; incomplete or nonresponsive when it does not give a checkable answer to the question asked, including a range with no single current figure or an answer about a different region; unavailable when there was no answer to score. A figure the answer clearly labels as a past rule, with the right dates, is treated as a correct statement about the past. A claim the answer volunteers beyond what was asked is scored only when it bears on the amount, the threshold, who qualifies, the dates or the scheme; detail about a different scheme is noted but does not change the class. Every answer was scored twice, by two different AI models working independently from the same saved captures and official pages with the published rubric; the two sets of scores were compared before any disagreement was resolved. Agreement before adjudication: 59 of 60 answer slots on the answer class, and 57 of the 58 answers that were actually scored (two slots had no AI Overview); 3 of 3 on the figures extracted from answers that carried one. The scorers were Claude Opus 5.5 and Claude Fable 5.1, two models from the same family. The owner of NestPath decided the one disagreement and six further scoring questions raised in the adjudication list; in all seven cases the decision accepted the recommendation drafted by scorer 1, and the reasons are published. The disagreement is counted in the agreement rate.

How we describe the size of a gap

We report differences in separate columns and never add them together: the difference in a grant amount; the difference in a price, value or income threshold; a claim about who qualifies that has no dollar value; a difference in percentage points; and, where it helps, an illustration with its assumptions written out. On this capture day the fields used were one grant amount difference ($15,000 understatement on the Queensland grant), two threshold differences ($3,000 and $5,000 understatement on the Help to Buy income limits) and three eligibility claims. No percentage point difference arose and no illustration is used. None of these is a measured cost to any buyer.

What is public and what is private

Public: the question list, the official answer for each question with its source and dates, the class of every answer, the separate difference fields, whether each answer cited NestPath, the agreement rate, the seven adjudication decisions with their reasons, the rubric and the checksums of the official evidence. Private: the raw responses from both capture services, the full scoring sheets with the transcribed answer text, and any screenshot. We do not reproduce AI Overview or ChatGPT text; what an answer said is paraphrased. The private files can be checked by a journalist or an official on request to hello@nestpath.com.au, on the condition that the AI text is not republished.

Limits

One day, one place, one method. Everything on this page describes what two systems returned on 3 November 2026 (UTC), between 14:11 and 14:24, with the location set to Australia and the device set to desktop. We captured one day, so we do not know how stable these answers are. We report what the systems said on that day; we do not say why they said it, and we do not say what they would say on another day, to another person, or on another question.

Thirty questions we chose. The questions are ours and several overlap (four read the same federal cap table; two are Queensland duty; two are the First Home Super Saver Scheme). The counts here are counts of these 61 answer slots, 59 of them scored. They are not an estimate of how often AI is right or wrong about first home buyer money in general, and they say nothing about AI products we did not test.

How ChatGPT was reached. The ChatGPT answers came through a third party capture service (DataForSEO), which submitted each question from its own environment and reported the model as “gpt-6” for all 30 answers, with web search left to the product’s default rather than forced. This is not a signed in consumer session and not a person typing at chatgpt.com. We did not test whether another route to the same product would give a different answer.

Who scored and who decided. Both scorers were AI models from one family (Claude Opus 5.5 and Claude Fable 5.1). They worked independently from the same saved captures and official pages, and agreed on 59 of 60 answer slots before any disagreement was resolved, which is 57 of the 58 answers actually scored. The owner of NestPath decided the one disagreement and six further scoring questions raised in the adjudication list; in all seven cases the decision accepted the recommendation drafted by scorer 1, and the reasons are published above. The scoring sheets contain the AI text and stay private.

Rules that moved. The change date is shown for every out of date answer. The three out of date Google answers each matched a rule that changed between 1 October 2025 and 1 July 2026.

Two sites could not be fetched by script. RevenueSA and nt.gov.au block automated requests. Their pages were read and saved as page text from an ordinary browser on 2 November 2026; four of the seven pages were read again in the browser on 3 November, and for the other three the 2 November text is the evidence. For those rows the saved text, not a server checksum, is the evidence.

Our interest. NestPath publishes guides on these topics, and one of the 61 answer slots cited a NestPath page. See the disclosure below.

Our interest

NestPath (nestpath.com.au) publishes first home buyer guides and calculators and earns referral income from some links on its pages. It has a commercial interest in people checking official sources and in its own pages being found. For that reason no NestPath page was used as a source for any official answer, and the table shows for every answer whether the AI cited NestPath (one did, the Google AI Overview on the New South Wales grant, which was scored correct). The owner of NestPath, Anish Puri, who is also the author of this page, decided the one scoring disagreement and six further scoring questions raised in the adjudication list, accepting in all seven cases the recommendation drafted by scorer 1. NestPath received no payment from any party named here, and no AI company was asked for comment before publication.

Official sources and checksums

Every official page used, with the date shown on the page, the machine readable date where one exists, and the md5 checksum of the file saved on 2 November 2026. Each page was checked on 2 November 2026 and 3 November 2026.

Every page was fetched again on 3 November 2026 between 14:11 and 14:12 UTC. Twenty six fetches returned the page. Seven, on two sites that block automated requests (RevenueSA and nt.gov.au), could not be fetched by script; four of those seven pages were read again in an ordinary browser on 3 November, and for the other three the page text saved from a browser on 2 November is the evidence. Of the 26 pages fetched, 17 had a different checksum from 2 November. For 16 the difference was page furniture such as menus and tokens. The Reserve Bank cash rate table had gained the 4 November 2026 row (4.85 per cent) after the 3 November decision, and that is the figure the cash rate question was scored against. No other official answer changed between the two checks.

The Reserve Bank announced the 3 November 2026 decision in its statement that day, saved on 3 November 2026 and listed above. The cash rate table saved at 14:23 UTC, which already carried the 4 November 2026 row, is the evidence the cash rate question was scored against.

Related reading

• Our sources list
• How NestPath researches and checks its pages
• How AI Overviews are swallowing first home buyer clicks
• Cited, not clicked