A page on the Spirit of SPOJ

OK, I did it, and I saw that the only proposed solution is the creation of a new problem set with no available solution.
As the only option, I have some questions:
* are all unspoiled classical problems aimed to be moved to the 'protected' set ? (I hope that)
* (else) are all classical problems aimed to be spoiled by publishing ? (I don't wish that)
* _____(or) will publishing be available only after the first AC ? (not perfect, but maybe acceptable)
* What about copyright issue : where are stored those 'papers' (will it be a spoj feature ?), what will be the license ? (very important questions)

Why not simply allow publishing on some selected tutorial tasks? This would work as a test. If conclusive it could lead to a next level with published solution.

Well, it won't be any option you mentioned, I think.
I think we shouldn't force anything. So moving all unspoiled problems to the new problemset and leaving all unspoiled problems in the classical problemset are both not very good solutions.
I think it would be better for Problem Setters to decide. So, we'll make a list of unspoiled problems, then make a set (not multiset) of PSs who created these problems and will ask them which problemset they'd prefer. If they don't answer I think that we'll move their problems to the new problemset (but we'll see how the survey goes).

And as for publishing solutions - everything would be the same. If someone wants to do it - fine, but not on SPOJ (at least for now, as there are many many more important features to implement and bugs to fix). But we'll inform them about the statistics of cheating, perhaps.

What do you think about this idea?

I've been writing something up and hopefully it could be useful.

A lot of this discussion arises from the fact that the score for classical problems is calculated based on number of solvers, specifically 80 / (40 + number_of_solvers). For this type of score system we have:

1) high granularity
2) objectivity (avoids subjective decisions on how many points a problem is worth)
3) harder problems tend to be worth more points
4) a problem's point value becomes negligible (very close to 0) when there are very many solvers
5) no domain knowledge needed, similar to general machine learning algorithms (no need to entrust anyone with the responsibility of deciding a problem's point value -- the system just "figures it out")
6) cheaters (users who submit code that they found and didn't write) lower point values of problems that they submit to
7) everybody's classical scores go down over time if they don't solve more problems, but the point values of older and harder problems are more stable than newer and easier problems

This is in contrast to other score systems, for example the Polish SPOJ rank table ( http://www.mariosoft.pl/spoj/rank.php4 ), where Easy problems are worth 1 point, Medium 2, Hard 3, Challenge 4.

Lately we have been discussing unfairness arising from point (6). Some of my ideas, observations, or opinions:

A) I think it's very good that the current system allows some solvers such as Min_25 to get very high rank by focusing more on harder problems and not as much on quantity of problems solved.
B) When cheaters lower a problem's point value, it does not affect the relative rankings between two people who both solved that problem or both didn't solve that problem.
C) It's not obvious how much cheating is actually occurring, and how much it is affecting point values. We don't have good data / statistics on that.
D) The formula was created many years ago, when SPOJ user base was smaller. It could be worth considering adjusting the formula to e.g. 120 / (60 + number_of_solvers) to counteract "inflation". Those are just example numbers I pulled out of thin air, I didn't do any analysis there.
E) Another possible modification to the score system: we could make it so that problems are worth at least e.g. 0.1 points no matter how many solvers there are. This could make doing the easier problems a seem little more worthwhile for beginners? For now, any problem that is solved by more than 760 people will be worth less than 0.1 points. (And I also didn't do any analysis to choose 0.1, maybe a different value could make more sense.)
F) Very rarely do cheaters rise to the very top ranks. The only ones I can remember are arena (Rully Soelaiman) and baladeva, and they were removed from the main rankings.
G) Even if cheaters undermine the point values, it does not change how interesting or uninteresting the problems themselves are.
H) The cheating issue bothers me a bit, but not very much. I'd rather spend my time working on new problems than worrying about such things. But it is good to see that the issue is being given serious consideration.
I) It seems reasonable to take into account that the SPOJ problem set is not the same as a live contest with a deadline and possibly prizes involved, where the rankings could be considered more "important" and cheating more closely tracked.
J) Instead of creating a new problem set, maybe it would be more interesting to try to do something like ALGOLIGA1 but in English (also similar to the DASM contests many years ago). (Francky and I discussed the idea in some previous email correspondence, but I hadn't thought about it for a while.)
K) I'm not convinced that we should hold publishers (of solutions or editorials) responsible for the cheating that occurs. I think the blame really rests on the cheaters, and the publishers are in some sense innocent. For publishers, I think that they should just try to be discreet if they can -- maybe write a note discouraging cheating, modify code or only give pseudocode so that it can't be copied and pasted, make it non-obvious to search for, or some things like that. Aside from that, it seems to make more sense to target cheaters and not publishers.
L) It is not practical to try to police the whole Internet. We have limited resources.
M) Giving any publisher the power to make a problem worth 0 points just by publishing a solution seems rather bad / silly. (This was stated nicely be numerix.)
N) Making any big changes that affect the rankings of many honest solvers negatively should be avoided if possible.

I know those are a lot of points. Point (J) is most relevant to the recent posts in this thread.

Well, my idea was to add an additional feature, as Francky suggested. There would be another problemset "unspoiled" and there would be another ranking for unspoiled problems. The old ranking would include both problems from "classical" and "unspoiled" problemsets. So I don't think anyone would be dissatisfied. Or am I not seeing something?

As for AlgoLiga, I don't understand how it solves the problem. Problems from current round would be unspoiled but later when they are moved to main SPOJ (would they be moved?) they can be spoiled again. So people would have limited time to solve some unspoiled problems. It is a good idea to make such a contest but just in parallel with the new problemset.
// And BTW, we made AlgoLiga because there was no regular Polish programming contest. And in English there is CodeForces where Problem Setters can even earn money for preparing problems for a round. But if someone wants to make such a contest I don't have anything against.

That sounds like it might work. I don't think those details were presented in the survey or this forum thread, so it was unclear what was being discussed. And I only offered mild criticism for what it vaguely sounded like. Now with the details, I do have a few hesitations:

a) It seems like it might help cheaters a little, by clearly identifying which problems are known to have published solutions and which are not.
b) It seems kind of unnecessary, but I don't know. The current ranking system still gives a pretty good indication of where people are in relation to each other, doesn't it? I'm just not sure if we gain a lot by doing it that way.

Well, "So people would have limited time to solve some unspoiled problems" is really the point I was making. That seems to be a big feature of live contests in general. And it also seems to apply to the proposed idea of adding a new problem set.

For comparison, Project Euler was the first problem solving contest site that I participated in, before I knew SPOJ existed. It was clear to me while participating on that site that the way to prove you were not a cheater was to get on the list of fastest solvers for several problems, if you could. (Of course, trying to solve those problems as fast is possible is also a fun and hard challenge, regardless of whether there are any cheaters there or not.)

With a contest like ALGOLIGA, aside from being interesting in itself, cheating could be monitored more closely, and then after deadline the results get locked in permanently, so there is no possibility to have the points diluted by cheaters at some future date. After deadline, problems could be added to main SPOJ without any official solutions posted, and treated like other classical problems.

We know that there are many honest problem solvers on SPOJ. (For evidence, just look at the status page and see how many non-ACs are getting submitted all the time.) But some may worry that their rank will not be taken very seriously because of the effect of cheaters, and that it may not be easy to tell whether or not you are one of them. While I think the main ranking is still a decently meaningful metric, having another contest with rounds and deadlines might be a nice alternative option for those who would be interested. (And maybe a way to generate some extra excitement in the SPOJ community, and encourage new, original and high quality problems to be published.)

It's just an idea that seemed to apply to the discussion. It doesn't really have to be considered in the light of cheating. It is really a separate idea that can stand by itself.

And I know it would take a lot of work to organise such a contest. I'm not suggesting admins would need to devote any time to it (besides possibly making announcements of new rounds coming up). It would only be an option if we happened to have enough skilled problem setters who wanted to see the project happen.

That's good to know. My knowledge of other programming contests is generally quite limited (aside from knowing their names).

Maybe the idea of a contest like ALGOLIGA but in English (with a similar connection to main SPOJ that ALGOLIGA has to Polish SPOJ) is redundant considering other contests like CodeForces, TopCoder, Google Code Jam, etc. I hadn't considered that much. I guess it would require looking at a lot of details, to see if there was anything really new to offer. (Although maybe it would be worth doing even if feels a little redundant, for various reasons like the effect on the community.)

Well, I don't see how it helps. If cheater wants to solve problem XYZ he just needs to google "spoj XYZ solution". So it's already as easy as possible. And as for the "unspoiled" problemset - banning will be not only for cheaters but for publishers as well. So it will be a holy ground.

That's why we did this survey. If there are not many users who would like such a problemset, it won't be implemented.

As for AlgoLiga, I understand now it's relevance to the topic but I still think that these ideas should be parallel. But it would be awesome to have regularly new problems coming to the new problemset.
So if you want, make another topic called "AlgoLeague" if you want to organize this contest. Some PSs should be interested in preparing some problems from time to time if we were able to find them only in Poland.

I'm not sure what I had suggested, in what context, but I don't feel good with a new problem set with 'protected' problems, as it would implies other are 'abandoned'
and psolvers would be encouraged to publish solutions (I read that somewhere).

I didn't see in survey a question like "Are opposed to publishing ?" or "Do you want to discourage publishing ?"
I know we can't force people to do what they want out of spoj, a basic freedom.
But I think we should discourage them as possible to publish, and forbid links in profile.
After that, if a cheater want to use google, he will find something he will think as learning (even if it's the opposite).

I will support anything in the way to protect the largest classical problems, split the set isn't a good solution imho.
I perhaps wrote some thoughts in the case (unhappily) where democracy conduced to split the set, it's not my favorite option.

Who is favorable to publish solutions ????????
Do we should let speak a lot of very young students who prefer facilities, or mainly the honorable high ranked 'old' users (like Blue Mary) as "the Guardian of the Spirit of SPOJ" ? (Voluntarily excessive style - please read "democracy is a complex stuff")

==

AlgoLeague is a parallel, independent, option, very nice, I will participate with pleasure to create problems if it exits.

But, but Francky... The idea of a new problemset was yours. I was hoping to satisfy mostly you and jbourne as other users didn't seem that's such big of a deal. Here is a quote from this topic (maybe I didn't understand something).

I also think that most users will solve first problems from the unspoiled problemset but I doubt that it will kill "classical". There will still be some interesting problems, users will still get points solving these problems (but they will count only in one ranking) and jbourne's 1st category will still solve problems from both problemsets. And as there will be much less problems in the new problemset people who manage to solve all of them (or all that they are able to) will go back to the classical problemset. I really believe that's this new problemset is not a death sentence for "classical". It won't become the second "tutorial".

Discouraging publishers won't take place as I've already asked about it. So knowing what is not an option I want to come up with a scenario that will satisfy most users. So forgive me for filtering your requests that I know can't happen. There is one thing that can happen, though. It's not much, but... You can discourage cheaters. By that I mean that you can write some users who publish their codes without describing the algorithm and explain your point of view. Because why wouldn't you be able to? Nobody can't forbid nobody to write anybody (this sentence is probably incorrect gramatically - I have problems with double/triple negations). I know that's time consuming and perhaps won't do much good but I really can't do anything else. Hope you understand.

As for Mitch's suggestions regarding the formula, I'll suggest your two improvements (0.1 minimum and different constants in the formula).

Thanks to Mitch for considering so many aspects. I agree with him in almost all of the aspects mentioned:
A) - G) and L) - M): absolutely!
H): It bothers me a bit more than a bit, but not too much. If I have time for SPOJ, problem solving and sometimes setting is more interesting than hunting cheaters.

K): Some of the aspects I also mentioned before (I'm not sure if it was in a private mail or in that thread). In my opinion that is a point that should get more attention. If I understood kokosek's post (below) right, admins are not willing to send emails to those users, that publish solutions, not even if there is a link to that publications on the user page. I cannot understand the reason for that, but I won't complain as they are doing a great job and will have enough work to do.
But as my own experience (I mentioned that in a former post) was a good one, it could really be worth spending more effort on that: Tell those publishers what happens with there published solutions, ask them friendly to stop that and offer them alternatives (Mitch mentioned some of them).
Instead of publishing AC code ...
... show pseudo-code
... give explanations on how to solve it resp. how they solved it incl. some insights they gained
... modify the AC code minimally in a way that it will not work, but will work if you correct that "error". That "error" must be of a kind that a real understandig of the algorithm is necessary to repair the code.
... just show a list of their solved SPOJ problems, perhaps with runtime. That could be a solution for those publishers that want to show what they are able to achieve in programming.

Mitch's suggestions for modifications of the ranking calculations are an interesting point. I have no concrete suggestion on how to change it, but e.g. the "minimum of 0.1 points anyway" (or some other kind of minimum value) sounds good.

Finally: I do not think a new "unspoiled SPOJ subcontest" is necessary, maybe not even helpful as new problems could arise.
Banning cheaters has been done more consequently recently than in the last 5 years as far as I can see. If those dedicated users (that e.g. contribute to that thread) and EB members in addition send emails to publishers, that could be enough.

@kokosek:
It was an answer at your proposition : spoj.com/forum/viewtopic.php ... cf7#p54209
I failed to distinguish a new section and a new contest, but I thought this answer shows many doubts.

I read somewhere :

I thought spoj could include a way to publish solutions, it's at the opposite (for me) of the spirit of spoj.
This frightened me.

==
from numerix:

I agree with that. If we can't discourage to publish out of spoj (a basic freedom, as I already state), we should be able to force this simple rule "No direct link to solutions within spoj ; (ie comments, forum, user pages)".

I should have included mention of a rule like "No direct link to solutions within spoj ; (ie comments, forum, user pages)" in my point (K). I support that also.