The President came to see me pretty early this morning in relation to
the exchange of prisoners. It had troubled him through the night. I was at no
loss to perceive that behind the subject of exchange there were matters
undisclosed to me. He read again this morning the closing remarks of a long
telegram from Butler. I have no question there were improper remarks in that
dispatch which they at the War Department were unwilling either Mr. Fox or
myself should see, for I called Fox in to have all the facts disclosed. He and
Webb had, by their correspondence, led to the late movement, which was,
however, humane and right. The President said he wanted the subject to be got
along with harmoniously, that they were greatly ruffled at the War Department,
and if I had no objection he would go and see Seward, tell him the facts, get
him to come over, and bring the Secretary of War and all in interest to a
consultation. I told him I had no objection, nor any feeling, as it affected
myself, on the subject. All I wanted was our imprisoned men.
In less than an hour the President returned with Seward. We went
briefly over the question and read to him Mallory's letter. After discussing
the subject, went, by request of the President, with him to the War Department.
General Hitchcock and General Halleck came in soon. Stanton was ill-mannered,
as usual, where things did not please him, and on one or two occasions a little
offensive. Did not know why there should be different exchanges; the Rebels
would not recognize negroes. I told him that, while general cartel was
neglected, the army were making exchanges here, and by Butler on the James,
Sherman at Atlanta, Canby at New Orleans, and Foster at Hilton Head. I thought
it proper and felt it my duty to see that the naval men were not entirely
neglected. That no question as regards color had ever come up in regard to
naval exchange; that colored men in our service were not a distinct
organization, etc., etc. It was, he said, our duty to prevent Rebel masters
from reclaiming slaves who had been in our service. He thought I ought not to
write the Confederate Secretary of the Navy, recognizing him as Secretary. That
the slave-owners would insist on retaining and reclaiming their slaves wherever
and whenever they could, I had no doubt. It was a question of property, and of
local and legal right with them which we could not prevent. It was a
complicated and embarrassing question, but he must not suppose, nor would the
country permit our countrymen to suffer in captivity on such a question. To
absolutely stop exchanges because owners held on to their slaves when they got
them was an atrocious wrong, one that I would not be a party to.
As regarded Mallory, I told him I had carefully avoided giving him a
title, — that I had written to the Hon. Mr. Mallory in answer to a
communication I had received.
The President said that the correspondence was a past transaction, that
we need not disturb that matter; the Navy arrangement must go forward, and the
Navy have its men. He wrote and read a brief letter to General Grant proposing
to turn over the prisoners we had sent to him. After reading it he asked for
comments and opinions. General Hitchcock, a man of warm sympathies but little
moral courage, began a speech, sycophantic to Stanton, intimating that the War Department
should have exclusive control of the cartel, etc. I told him I was perfectly
willing and desired it, if they would not obstruct the exchange but get back
our men. All assented to the President's letter. Stanton and Seward preferred
it should be addressed to General Butler instead of General Grant, but the
President preferred addressing the General-in-Chief and I commended his
preference. We telegraphed Capt. Melancthon Smith, to turn the prisoners over
to General Grant to be disposed of.
In the course of the conversation, Stanton, who began to feel that his
position might not stand, said he had known nothing about these exchanges. I
told him we had written him requesting that the Rebel prisoners at different
points might be sent to Fort Warren in order to be exchanged. General
Hitchcock, his commissioner, had been consulted in the matter, and had
communicated with Mr. Fox, to whom had been given the charge of details for the
Navy, as General Hitchcock had them for the War Department. General Hitchcock
himself had proposed that we should take some one or two army men on board the
Circassian as a special favor. After this matter was disposed of, and before
leaving the room, Seward spoke aside to the President and also to the Secretary
of War, stating he had appointed a meeting between them and Weed and Raymond,
who were in the building, he had no doubt. As I came out of the Secretary's
apartment, Weed was in the opposite room, and evidently saw me, for he
immediately stepped aside so as not to be seen. It was not an accidental move,
but hastily and awkwardly done. They waited half behind the door until we
passed out.
SOURCE: Gideon Welles, Diary of Gideon Welles, Secretary of the
Navy Under Lincoln and Johnson, Vol. 2: April 1, 1864 — December 31, 1866,
p. 169-72
