Prepare grill. In a food processor pulse olives until chopped fine. Add remaining relish ingredients and pulse until olives are minced. Brush both sides of swordfish with oil and season with salt and pepper. Grill fish on a rack set 5 to 6 inches over glowing coals 4 minutes on each side, or until just cooked through. (Alternatively, fish may be grilled in a hot well-seasoned ridged grill pan over moderate heat.) Serve swordfish topped with relish.