In a report, the State Comptroller identified "serious shortcomings" in the way decisions were taken by Mr Netanyahu before the May 2010 incident.
The activists were killed in clashes with commandos who boarded the lead flotilla vessel, Mavi Marmara.
The incident strained ties with Turkey.
Turkey has demanded Israel apologise for what it said was an unjustified use of violence, and expelled Israel's ambassador.
Israel says its commandos used live fire only after being attacked with clubs, knives and guns.
The Free Gaza Flotilla, which had more than 600 pro-Palestinian activists aboard several aid ships, was trying to break Israel's blockade of Gaza at the time.
In his report, State Comptroller Micha Lindenstrauss said "substantive and significant deficiencies were discovered in the decision-making process... that Prime Minister Benjamin Netanyahu led and oversaw".
Mr Lindenstrauss criticised the prime minister for consulting only with his defence and foreign ministers, rather than a wider circle of officials with a view to formulating a policy about what to do.
"The process of decision-making was done without orderly, agreed-upon, co-ordinated and documented staff work, despite the recognition of the senior political echelon and IDF (Israel Defence Forces) chiefs, intelligence bodies and the National Security Council on the exceptional nature of the Turkish flotilla compared to previous flotillas," the reports said.
The BBC's Wyre Davies in Jerusalem says despite the numerous faults found in this report, it is unlikely to adversely affect Mr Netanyahu's standing amongst Israelis, most of whom supported the action against the Mavi Marmara.
It could, though, prompt changes in Israeli planning for future confrontations - in particular with regard to Israel's tense relations with Iran, he adds.
Mr Netanyahu's office responded to the findings, saying Israelis were "enjoying a level of security not seen for many years... the direct result of responsible management and determined policy."
The report is the latest of several Israeli and international inquiries into the Mavi Marmara incident.
Last year, an Israeli government-commissioned inquiry found the actions of the Israeli navy in dealing with the flotilla and Israel's naval blockade of Gaza were both legal under international law.
Turkey's own inquiry said the incident was a violation of international law, "tantamount to banditry and piracy", and described the killings as "state-sponsored terrorism".
A separate UN inquiry found the Israeli commandos' actions were "excessive and unreasonable".
However, it also found Israel's naval blockade "was imposed as a legitimate security measure" which "complied with the requirements of international law".
Israel says the blockade is necessary to prevent weapons and ammunition being smuggled to the Gaza Strip, which has been ruled by the Palestinian militant group Hamas since 2007.