Overlaping of html and css based boxes

Assuming you have full control of HTML and CSS, and I have predicted the result you are going for, here's how I would go about it.

http://jsfiddle.net/9b6YS/

HTML

<div class="signin">
    <a href="#loginmodal" id="modaltrigger">Signin</a>
    or 
    <a href="signup.html">Signup</a>
</div>

CSS

.signin {
    display: inline-block;
    background: #1371B3;
    border-radius: 30px;
    color: #fff;
    font-size: 0.9em;
    font-weight: bold;
}
.signin a {
    display: inline-block;
    padding: 10px;
    margin: 0;
    border-radius: 10px;
    color: #fff;
    text-decoration: none;
}
.signin a:hover {
    color: #76AE41;
}

Note that I dropped the complexity. It doesn't seem to add anything, but perhaps you know something I don't.

If you wolud like them overlapped and on a single line then I would recommend not using float in this instance and instead use display: inline-block

DEMO http://jsfiddle.net/y4TA2/4/

.signin li {
    display: inline-block;
    text-align: middle;
    height: 30px;
    margin-left: -10px;
}

Your CSS for the class "a" is targeting the H6 or box because you said you want to target the LI with the class of "a".

Your first "sign" LI in button has no class. Try changing it to:

<li class="a">
    <a href="#loginmodal" id="modaltrigger">
        <h6>Signin</h6>
 </li>
 <li >
     <h6>or</h6>
 </li>

Click For Demo

HTML

<ul>
    <li> <div class="signin"> <span >sign in</span> </div></li>

    <li><div class="Or"> <span > or </span> </div></li>
    <li><div class="signOut"> <span >sign Out</span> </div></li>
</ul>

CSS

.signin{

    background-color:#1371B3;
    height:25px;
    width:50px;
        border-top-left-radius:10px;
      border-bottom-left-radius:10px;
     text-align:center;    
     z-index:2;    
      margin-right:-3px;   
    border:1px solid #0c456e;
}
.signin span{

    padding:5px;
    color:#fff;
}
.Or{
        background-color:#1371B3;
    height:35px;
    width:50px;
        border-radius:10px;
    margin-top:-5px;

 text-align:center;   
     z-index:1;    
       border:1px solid #0c456e;a
}
.Or span{

    padding:5px;
    color:#fff;

}
.signOut{

    background-color:#1371B3;
    height:25px;
    width:50px;
    border-top-right-radius:10px;
      border-bottom-right-radius:10px;
 z-index:2px;    
      margin-left:-3px;    
        border:1px solid #0c456e;
}

.signOut span{

    padding:5px;
    color:#fff;
     text-align:center;  

}
ul li{
    float:left;
    list-style:none;

}

after reading all the answers here, I think you are going for something like this:

http://jsfiddle.net/s6KF8/2/

<div class="signin">
<ul>
  <li>
    <a href="#loginmodal" id="modaltrigger" class="login">
      <h6>Signin</h6>
    </a>
  </li>
  <li class="or_circle">
    <h6>or</h6>
  </li>
  <li >
    <a href="signup.html" class="signup">
      <h6>Signup</h6>
    </a>
  </li>
</ul>
</div>

and for css

.signin{
    vertical-align: middle;
    position:relative;
}
.signin h6 {
    margin:0;
}
.signin li {
   list-style:none; 
   float:left;
   text-align: center;
   font-size: 0.9em;
   color: #ffffff;
   vertical-align: middle;
}

.signin a {
    color:#fff;
    display:inline-block;
}

.login {
    border-radius: 2em 0 0 2em;
    padding: 5px 15px;
    background: #1371B3;
    margin-right:-10px;
    margin-top:8px;
}
.signup { 
    border-radius: 0 2em 2em 0;
    padding: 5px 15px;
    background: #1371B3;
    margin-left:-15px;
    margin-top:8px;
}

.or_circle{
    border-radius: 2em;
    padding: 1em;
    background: #1371B3;
    position:relative;
}

a.signup:hover,a.login:hover {
    background: #76AE41;
}