What is the difference between lower bound and tight bound?

Question

With the reference of this answer, what is Theta (tight bound)?

Omega is lower bound, quite understood, the minimum time an algorithm may take. And we know Big-O is for upper bound, means the maximum time an algorithm may take. But I have no idea regarding the Theta.

Brantley

Answer 1

Big O is the upper bound, while Omega is the lower bound. Theta requires both Big O and Omega, so that's why it's referred to as a tight bound (it must be both the upper and lower bound).

For example, an algorithm taking Omega(n log n) takes at least n log n time, but has no upper limit. An algorithm taking Theta(n log n) is far preferential since it takes at least n log n (Omega n log n) and no more than n log n (Big O n log n).

Whistler

Oh.. Now the term "tight bound" appearing quite self-explaining to me. Thanks Chris. Stupid me, perhaps I was expecting some complex idea. :) – Brantley

Yea, there's a lot of fancy notation thrown around but it's not too complex once you get it under your belt. – Whistler

This freely available document from Virginia Tech explains with examples the differences in performance between algorithms of different complexities and briefly explains Asymptotic Analysis: people.cs.vt.edu/shaffer/Book/C++3e20120102.pdf – Spirogyra

What do you mean by " An algorithm taking Theta(n log n) is far preferential since it takes at least n log n (Omega n log n) and no more than n log n (Big O n log n). ", as in, is it the exact complexity of a algorithm as you wrote said at least Omega(nlogn) and at max BigO(nlogn) ? – Clinkstone

@NikhilVerma Doesn't bill-the-lizard's answer below cover you? – Varden

In simple terms can we call: upper bound (Big(O)) as the worst case? tight bound as the average case? lower bound (Omega) as the best case? – Mattoid

Answer 2

Θ-notation (theta notation) is called tight-bound because it's more precise than O-notation and Ω-notation (omega notation).

If I were lazy, I could say that binary search on a sorted array is O(n²), O(n³), and O(2ⁿ), and I would be technically correct in every case. That's because O-notation only specifies an upper bound, and binary search is bounded on the high side by all of those functions, just not very closely. These lazy estimates would be useless.

Θ-notation solves this problem by combining O-notation and Ω-notation. If I say that binary search is Θ(log n), that gives you more precise information. It tells you that the algorithm is bounded on both sides by the given function, so it will never be significantly faster or slower than stated.

Argali

If I were lazy, I could say that binary search on a sorted array is O(n2), O(n3), and O(2n), and I would be technically correct in every case - It seems most of the people in computer world are lazy only as everyone mostly talks about Big O complexities only. – Location

If I were lazy, I could say that binary search on a sorted array is O(n2), O(n3), and O(2n), and I would be technically correct in every case In case someone is confused with this: For that kind of functions which are not asymptotically tight small-o notation is used. Example: - The bound 2n^2 = O(n^2) is asymptotically tight, but the bound 2n = O(n^2) is not. Read more: #1364944 – Beyond

If this is the case, why do we use Big-O more often? Is it because it is easier to calculate by hand? – Bolo

Answer 3

If you have something that's O(f(n)) that means there's are k, g(n) such that f(n) ≤ k g(n).

If you have something that's Ω(f(n)) that means there's are k, g(n) such that f(n) ≥ k g(n).

And if you have a something with O(f(n)) and Ω(f(n)), then it's Θ(f(n).

The Wikipedia article is decent, if a little dense.

Lucic

Now reading the family of Bachmann-Landau notations. Thanks Charlie, I went there before, but returned without proceeding to its length. – Brantley

Hey, it's good to get a refresh on doctoral comps every so often. – Lucic

Notice that Landau's big-O notation is not limited to algorithmic complexity. – Lucic

This looks wrong. In the first line it should read "If you have something that's O(g(n))", that is, g instead of f, and the rest can be left as it is. Same goes for the second line: it should be "If you have something that's Ω(g(n))". Can you please double check? – Stockdale

The whole topic is so messed up that someone with that credite also might get it wrong :D Joking aside, someone needs to fix this answer. This confuses people (it did me very much). – Loralorain

Did you perhaps mean to say "If you have f(n) = O(g(n)), there's a k, g(n) such that f(n) ≤ k g(n)"? – Carmel

Answer 4

Asymptotic upper bound means that a given algorithm executes during the maximum amount of time, depending on the number of inputs.

Let's take a sorting algorithm as an example. If all the elements of an array are in descending order, then to sort them, it will take a running time of O(n), showing upper bound complexity. If the array is already sorted, the value will be O(1).

Generally, O-notation is used for the upper bound complexity.

---

Asymptotically tight bound (c₁g(n) ≤ f(n) ≤ c₂g(n)) shows the average bound complexity for a function, having a value between bound limits (upper bound and lower bound), where c₁ and c₂ are constants.

Balthazar

if the array is sorted, the bound will still be O(n) – Rundell

@ArunAravind Can you explain why? – Champerty

Answer 5

The phrases minimum time and maximum time are a bit misleading here. When we talk about big O notations, it's not the actual time we are interested in, it is how the time increases when our input size gets bigger. And it's usually the average or worst case time we are talking about, not best case, which usually is not meaningful in solving our problems.

Using the array search in the accepted answer to the other question as an example. The time it takes to find a particular number in list of size n is n/2 * some_constant in average. If you treat it as a function f(n) = n/2*some_constant, it increases no faster than g(n) = n, in the sense as given by Charlie. Also, it increases no slower than g(n) either. Hence, g(n) is actually both an upper bound and a lower bound of f(n) in Big-O notation, so the complexity of linear search is exactly n, meaning that it is Theta(n).

In this regard, the explanation in the accepted answer to the other question is not entirely correct, which claims that O(n) is upper bound because the algorithm can run in constant time for some inputs (this is the best case I mentioned above, which is not really what we want to know about the running time).

Circumfluous

So, can we say that Ω is the best case, and O is the worst?. . .. and should we replace the terms as best case, and worst case, respectively? – Brantley

Best case is O(1) for any problem? – Frantz

@Adeel, no, Theta and O can both refer to either average case or worst case. @Zach, well, not exactly. Thanks for pointing that out. – Circumfluous

Answer 6

Precisely the lower bound or $\omega $ bfon f(n) means the set of functions which are asymptotically less or equal to f(n) i.e U g(n)≤ cf(n) $\for all $`un≥ n' For some c, n' $\in $ $\Bbb{N}$

And the upper bound or $\mathit{O}$ on f(n) means the set of functions which are assymptotically greater or equal to f(n) which mathematically tells,

$ g(n)\ge cf(n) \for all n\ge n' $ , for some c,n' $\in $ $\Bbb{N}$.

Now the $\Theta $ is the intersection of the above written two

$\theta $

Like if a algorithm is like " exactly $\Omega\left( f(n)\ right$ " then it's better to say it's $\Theta\left(f(n)\right)$ .

Or , we can say also that it give us the actual speed where $ \omega $ gives us the lowest limit.

Protomorphic

Answer 7

The basic difference between

Blockquote

asymptotically upper bound and asymptotically tight Asym.upperbound means a given algorythm that can executes with maximum amount of time depending upon the number of inputs ,for eg in sorting algo if all the array (n)elements are in descending order then for ascending them it will take a running time of O(n) which shows upper bound complexity ,but if they are already sorted then it will take ohm(1).so we generally used "O"notation for upper bound complexity.

Asym. tightbound bound shows the for eg(c1g(n)<=f(n)<=c2g(n)) shows the tight bound limit such that the function have the value in between two bound (upper bound and lower bound),giving the average case.

Conwell

You shouldn't answer old questions if your answer doesn't add anythin g to already accepted answers. – Kutzenco