The simple answer is that the Nazi plan wasn't always to exterminate the Jews, Romani,
et al en masse. The plan, originally, was to get them out of German land and the
Lebensraum. This is why a) Jews were allowed, and indeed
encouraged to emigrate in the early stages of Nazi rule. Boycotts, prohibitions against Jews owning land or operating businesses, restriction of citizenship eligibility, pogroms, and ghettos were intended as measures to make life so intolerable for the Jews that they would view leaving Germany as preferable. This plan was reasonably effective: by 1939, over half of the Jewish population in Germany had left Germany.
An initial plan was to forcibly relocate the remaining Jews to Madagascar, the idea being to create an ethnostate for Jews in a land so unaccommodating that the Jewish population would never be able to succeed and would languish forever in Africa, out of sight and out of mind. The plan was to defeat Great Britain via Operation Sea Lion, and use the captured British merchant fleet to effect the relocation, however, upon the failure of Sea Lion, the plan was abandoned.
With the capture of Poland, and the addition of 2 million Jews to Nazi-controlled lands, the plan shifted towards relocating the Jewish population to select ghettos in urban areas. Again, the plan was not necessarily extermination, but rather to restrict the Jewish population to certain "reservations," where the Jewish population could be contained and, potentially, relocated (either to Madagascar or Siberia) at a later date via rail. In this, the Nazis obviously drew inspiration from the 19th century US Indian policy: keep the population confined select, reservations, thereby freeing up the good land for your own population to and keeping the undesirable population confined to specific areas where they won't be able to build political or economic power, won't be able to intermix with your own population, and if they die in the process, all the better. You also gain the benefit of having a documented, easily accessible, exploitable slave population should you need it. Described in such a way, it's possible for this treatment to sound rather quaint, but make no mistake, this is not my intention. Although the ostensible goal wasn't to
murder ghettoized populations
per se, death was nevertheless an inevitable consequence of this policy: conditions and treatment were appalling, and at least half a million Jews starved to death in the ghettos and forced labor camps. Extermination in this case wasn't the
de iure objective, but it was certainly the
de facto consequence of the policy.
German policy towards "The Jewish Question" shifted between 1939 and 1942. The first event that triggered this shift was
Kristallnacht . On November 7th, 1938, a German diplomat, Ernst vom Rath was assassinated in Paris by Herschel Grynszpan, a Polish Jew. The Nazi government used this assassination as a pretext to carry out an appalling series of pogroms. Prior to
Kristallnacht, the Nazis began a more marked effort to relocate their Jewish population to concentration camps, which initially had largely been used as holding and forced-labor camps for political dissidents. Again, the intention here wasn't initially to exterminate Jewish populations
en masse, but rather to keep them contained in select areas. Again, this wasn't quaint, pleasant, or peaceful, even within the context of how prisoners are ordinarily treated anywhere. Conditions were appalling - intentionally so. The
objective wasn't to kill them immediately, but to keep them confined in one place, and if they should die in the process, all the better.
The
Wannsee Conference, held in early 1942, was the point at which Nazi objectives shifted significantly. Hitler had made repeated references in the past to a destruction of Jewish populations should a world war break out, and, upon the Nazi declaration of war against the United States, Hitler intended to make good on his declaration. Following
Wannsee, the plan shifted from confining Jewish populations to ghettos and concentration camps, and restricting rights such that Jews would feel compelled to emigrate. The new plan was:
a) extract as much economic labor from the population as possible
b) once the individual was no longer able to provide economic labor, they were to be killed.
Reinhard Heydrich, the
SS-Obergruppenführer laid this plan out very explicitly:
Under proper guidance, in the course of the final solution the Jews are to be allocated for appropriate labor in the East. Able-bodied Jews, separated according to sex, will be taken in large work columns to these areas for work on roads, in the course of which action doubtless a large portion will be eliminated by natural causes.
The possible final remnant will, since it will undoubtedly consist of the most resistant portion, have to be treated accordingly because it is the product of natural selection and would, if released, act as the seed of a new Jewish revival (see the experience of history.) In the course of the practical execution of the final solution, Europe will be combed through from west to east. Germany proper, including the Protectorate of Bohemia and Moravia, will have to be handled first due to the housing problem and additional social and political necessities. The evacuated Jews will first be sent, group by group, to so-called transit ghettos, from which they will be transported to the East.
The death camps of the Holocaust were constructed beginning at the end of 1941. It was from this point that the objective became the immediate and total eradication of undesirable populations, principally Jews. Even here, the plan didn't call for the immediate murder of everyone. Upon entry to the concentration camps, 75% percent of the interned population were deemed unfit for labor and killed immediately, the other quarter were designated for slave labor: i.e. to be forced to work until they died, whether from starvation, exhaustion, disease (typhoid and dysentery were especially rampant, again often intentionally so), or the absolutely brutal treatment they received at the hands of camp guards.
As it became increasingly apparent that the Nazis were not going to win the war, efforts to exterminate the remaining population intensified. In 1944 up to 6,000 Jews were being murdered a day in Auschwitz alone. As the Soviets advanced towards the camps, commanders forcibly relocated inmates of the camps, forcing them to ride on open train cars for days with no food or water, or forcing them to walk the entire distance to camps closer to German borders, shooting anybody who lagged or fell behind.
In other words, to answer your question:
a) The Nazi plan wasn't always to exterminate the entire Jewish, Romani, et al. populations wholesale. Initially the plan was to force them to emigrate elsewhere. When this plan didn't work, the plan shifted towards
forcibly relocating them elsewhere (initial plans were first to Madagascar, then to Siberia). When this plan didn't work, the plan shifted towards forcibly confining these populations in ghettos and concentration camps.
b) Upon the Nazi declaration of war against the Americans, Hitler began to implement the "Final Solution", that is, the utter eradication of all Jews, Romani, etc. from Europe.
This is the point in which the plan became, as you said, "to just kill them all at once". Even still, the plan was to murder most of them and work the rest to death. As the Nazis began losing territory, the plan shifted again from "murder most of them" to "murder all of them"
c) Make no mistake though, even given these caveats, the holocaust, i.e. the period following the implementation of the "Final Solution" was brutally effective: 90% of Poland's Jewish population was murdered, 70% of the Jewish populations in Greece, Yugoslavia, Hungary, Lithuania, Bohemia, the Netherlands, Slovakia, and Latvia were murdered. 50% of the Jewish populations in Belgium, Romania, Luxembourg, Norway, and Estonia were murdered. ~33% of Jews living in Soviet land were murdered. 25% of the Jewish population in France was murdered. We don't really have accurate figures of just how many Romani were murdered, since their place in the history of the holocaust was ignored until the 1980s, and solid records for their populations have not and still are not particularly well documented, but an estimated 220,000-1.5M Romani were murdered.
So to
really answer your question: "why didn't they just kill them all at once:" they did.