When you are utilizing an SQL statement that you are attempting to insert a value which contains an apostrophe, also known as a single quote, you must backslash it so that you duplicate it. This allows the database to know the end of the string is not the apostrophe itself but a part of the string.
Here is the correct way you should write your SQL insert statement:
INSERT INTO Person (First, Last)
VALUES ('Joe', 'O''Brien');
In the example above, the apostrophe in O'Brien is escaped as O''Brien. That informs SQL that it's part of the string, rather than the end of the value.