Former Pennsylvania Sen. Arlen Specter, who died Sunday of cancer at age 82, leaves a complex political legacy behind – one that has flummoxed both parties in a career that spanned nearly 50 years.

A fixture in some of the most divisive and closely-watched events of his time, Specter perhaps had the biggest and longest-lasting impact on the Supreme Court. By shaping the court's membership, Specter indirectly influenced issues from abortion to racial preferences to the death penalty.

Slideshow: Arlen Specter: 1930 - 2012

Tom Williams / Roll Call

The Republican-turned-Democrat, who played a key role in many Supreme Court nominations, was 82.

Launch slideshow

But Specter’s unique place along the political spectrum frustrated both political parties over the decades.

In 1987, Specter helped keep one pioneering conservative, Robert Bork, off the high court.  In subsequent years, he helped shepherd three others to confirmation, including the controversial appointment of Justice Clarence Thomas.

Related: Andrea Mitchell remembers Arlen Specter

In 1991, his tenacious questioning of Anita Hill, who had accused conservative nominee Thomas of sexual harassment, helped blunt the impact of Hill’s testimony and played a role in Thomas’s eventual confirmation.

And, in his role as chairman of the Senate Judiciary Committee, Specter smoothed the path for confirmation of Chief Justice John Roberts in 2005 and Justice Samuel Alito in 2006.

But in 1987, Specter’s vote against the Bork nomination infuriated Republicans and conservatives, setting off an uncomfortable relationship between the senator and his party that lasted until the end of his career.

Originally a Democrat who turned Republican in the 1960s, when he served as district attorney of Philadelphia, Specter was elected five times to the Senate – a remarkable accomplishment for a Republican in that Democratic-leaning state.

Despite his support for Thomas, Alito and Roberts, Specter supported abortion rights and broke with his party not only on social issues but in his support for organized labor. He was one of only three Republican senators to vote for President Barack Obama’s stimulus plan in 2009.

Facing a tough primary challenge from conservative Pat Toomey in 2010, Specter left the GOP to run for re-election as a Democrat.

"I am not prepared to have my 29-year record in the United States Senate decided by the Pennsylvania Republican primary electorate - not prepared to have that record decided by that jury," Specter said, explaining his exit from the GOP.

In his appearance on NBC’s Meet the Press on May 3, 2009, after he’d switched parties, Specter told David Gregory, “My core views are freedom, a woman's right to choose, (I) consistently voted for increasing the minimum wage, for expanding unemployment compensation, for the nuclear test ban treaty, where I broke with Republicans.” 

He added that he got into politics because of his father, a Russian immigrant, who was a veteran of World War I. 

“The government broke the promise to pay World War I veterans a bonus. And Harry Specter was a little guy,” he said. “And you take a look at my record in the Senate, or my record in public life generally, I've always been for the little guy. I say in a sense that I, I'm on my way--I was on my way to Washington to get my father's bonus. I haven't gotten it yet, so I'm running for re-election.” 

He also criticized his Republican critics for singling out one or two votes that they disagreed with. 

“I voted 10,000 times. I don't expect people to agree with all my votes,” he said. “I don't agree with all of them at this time. But can you imagine picking one vote out of 10,000 and having the party say to me, in effect, ‘We don't want you as our candidate’? So there has to be room for people who are, who are moderates. It has to be Reagan's big tent again.” 

He was backed by Obama but Specter lost the 2010 Democratic primary to Rep. Joe Sestak, who in turn lost to Toomey.

In a statement Sunday, Toomey called Specter, “a man of sharp intelligence and dogged determination, Sen. Specter dedicated his life to public service and the commonwealth of Pennsylvania. His impact on our state and public policy will not be forgotten.”

Conservatives with long memories resented Specter for his vote against Ronald Reagan's Supreme Court nominee Bork in 1987. “There is nothing Arlen Specter could say that we would trust,” said Jan LaRue, general counsel of Concerned Women for America, a conservative advocacy group, said in 2004 when Specter took the helm of the Judiciary Committee.

But he compensated for his vote against Bork by his role in aiding Thomas, Roberts and Alito. 

At the 2006 annual meeting of the conservative lawyers group, the Federalist Society, Specter got a warm welcome. "I conclude that most of that applause is for Chief Justice Roberts and Justice Alito," he told the crowd.

He added that the confirmations of Roberts and Alito "may turn out to be the highlight" of Bush's presidency.

In 2004, when Specter – still a Republican at that point – faced a primary challenge from Toomey, President George W. Bush went to Pennsylvania to campaign for Specter. "He's a little bit independent-minded sometimes. But there's nothing wrong with that," Bush told Pennsylvania voters. "I can count on this man ... He's a firm ally when it matters most."

Early in his career, Specter served as assistant counsel to the Warren Commission, which investigated the assassination of President John F. Kennedy. Specter developed the hotly disputed “single bullet” theory of the shooting which contended that one bullet fired from Lee Harvey Oswald’s rifle had killed Kennedy and wounded Texas Gov. John Connolly, who was riding in Kennedy’s open limousine in Dallas on Nov, 22, 1963.