Clarification on bias of a perceptron

Question

Isn't it true that if a bias is not present, a line passing through origin should be able to linearly separate the two data sets??

But the most popular answer in this -->> question says

y                             
^                              
|  - + \\  +                   
| -    +\\ +   +               
| - -    \\ +                  
| -  -  + \\  +                
---------------------> x       
    stuck like this            

I am confused about it. Do you mean the origins in figure above are somewhere in middle of x-axis and y-axis? Can somebody please help me and clarify this?

Ethnarch

depends on your concept of line. if your talking about a straight line no you can't as in the case of the perceptron. your example shows you that. if it is a curved line as in a neural network (e.g. multilayer perceptron) then you can. – Footman

Answer 1

Alright, so perhaps the original ASCII graph was not 100% accurate! Let me try to depict this again:

            y                                    y
            ^                                    ^
   - + \\   | +                       -\\+       | +
 -     +\\  | +   +                 -   \\ +     | +   +
 - -     \\ |  +                    - -  \\      |  +
 -  -   + \\|    +                  -  -  \\+    |    +
------------------------> x        ---------------------------> x
    - -     |\\    +                   - - \\    |      +
 -   -    + | \\ +                  -   -   \\ + |    +
  -    - -  |  \\  + +               -    - -\\  |      + +
 -- -    -  |  +\\     ++           -- -    - \\ |  +       ++

     stuck like this                  needs to get like this
          y = ax                            y = ax + b
     (w0*x + w1*y = 0)               (w0*x + w1*y + w2*1 = 0)

Reckoning

Answer 2

I think your intuition is correct on this issue:

Do you mean the origins in figure above are somewhere in middle of x-axis and y-axis?

In my reading of the graph, yes.

I think the ASCII graph, as cool as it is, is a bit confusing here, because it shows a line that is not traveling through what would normally be considered as the origin. Normally one would think of the intersection of the x- and y-axis lines as the origin, but in this diagram the separating line is clearly not passing through said intersection. As you've noted, a perceptron without a bias term can only define a separating line that passes through the origin, so the ASCII graph must have some sort of odd origin that is floating out in space somewhere.

Also, note that a standard perceptron always defines a linear separator, but a linear separator is not guaranteed to be able to partition a given dataset correctly -- that depends completely on the dataset. There are also variants of the perceptron that use the "kernel trick" to define nonlinear separators, but that's a whole different story. :)

Hope that helps.

Volta

+1 you're right of course. I posted a new version :) – Reckoning