New site development update - Jan 2010

Announcements about the database, website, and plugins.
szsori
Site Admin
Posts: 1971
Joined: Fri Nov 03, 2006 5:23 pm

New site development update - Jan 2010

Postby szsori » Thu Jan 14, 2010 4:19 am

Just wanted to give people an idea where the new site is and what I've been working on. We've basically pinned down the design of the site (colors, graphics) and I've been able to make a style sheet that should handle 90% of what we need. Since bandwidth usage is a big priority on the new site, I've been working to keep the entire style sheet under 10kb and have so far been successful.

I also decided to go with jQuery instead of Prototype. I've been using Prototype for my other side project (greenmartini.com - possibly NSFW) and am comfortable with it, but really liked some of the plugins and core functionality of jQuery better. The result is that my actual development is going much more slowly right now, but over time it should speed up. I've decided to use jQuery Tools, which allows us to link the entire jQuery code with their plugins from their CDN, which saves us over 50kb per uncached load. Hurrah. Obviously that applies to the frontend only.

I've been focusing on the core functionality of the site to this point, which includes things like security, site navigation, and proper implementation of Smarty (the template system we're using). Now that I'm comfortable with that, I've been starting into site features including a developer's blog, nicer API documentation, and browsing by genre.

Most of you know that we're switching to MS SQL for the new site, which is going great so far. The site itself runs a bit slowly right now due to being on my overloaded work server, but even more because I haven't created any indexes, relationships, or other optimizations yet. Those things should help immensely.

My next tasks include finalization of the home page, "create" functionality for the blog system, and record versioning.

We're still a ways off on it, but I'm planning on dedicated 30 minutes a day to development which means that we should have something to show off "soon". The new site will be rolled out without all of the new features implemented, but with the new framework and database it should be far easier for us to add those things as time goes on.

Thanks for everyone's patience on this, especially the mods. I know our "tools" suck right now and I'm hoping we can make things immensely more easy for you folks. Also thanks to everyone who has been patient with our recent server issues. Hopefully they're worked out for now, but the silver lining is that they nudged me back into development. :)

p2mb2
Posts: 4
Joined: Wed Jan 13, 2010 4:44 pm

Re: New site development update - Jan 2010

Postby p2mb2 » Thu Jan 14, 2010 4:02 pm

keep it going ;) and if you need some programming help just shout it ;)

DJMcTom
Former Moderator
Posts: 17
Joined: Sun May 11, 2008 9:14 am

Re: New site development update - Jan 2010

Postby DJMcTom » Sat Jan 23, 2010 7:31 am

Is the API format going to change much/at all?
I'm about to start something fairly large using thetvdb, and it would kinda suck if you change all your indexes and apis after i've finished it.
On my current project the only tvdb index I use is the showid for fetching things, everything else I store in my own way. I was intending to use your own indexes for things so I can keep track of changes more accurately and match files to actual episodes on thetvdb rather than just rematching a file's season/ep to whatever fits after a show update.

I guess my question is: are you going to remove or change any fields(especially indexes) from the api?

szsori
Site Admin
Posts: 1971
Joined: Fri Nov 03, 2006 5:23 pm

Re: New site development update - Jan 2010

Postby szsori » Sat Jan 23, 2010 9:06 am

IDs on series will remain the same indefinitely. Episode ids will probably not remain the same. The new API, when we get to it, will be different but we'll also be supporting the existing format for a while as well. I'm guessing we'll look at lightweight alternatives to XML at that time too, like JSON. So in a nutshell, we'll be changing some things but will continue to support apps using the current system for quite a while.

Sypher
Posts: 33
Joined: Mon Jun 30, 2008 12:44 pm

Re: New site development update - Jan 2010

Postby Sypher » Wed Jan 27, 2010 6:49 am

szsori wrote:IDs on series will remain the same indefinitely. Episode ids will probably not remain the same. The new API, when we get to it, will be different but we'll also be supporting the existing format for a while as well. I'm guessing we'll look at lightweight alternatives to XML at that time too, like JSON. So in a nutshell, we'll be changing some things but will continue to support apps using the current system for quite a while.


Changing episodeid's will most definitely cause problems in applications that rely on those.
What i'd like to know is why you will be changing episodeid's but keep the showid's intact? :)

szsori
Site Admin
Posts: 1971
Joined: Fri Nov 03, 2006 5:23 pm

Re: New site development update - Jan 2010

Postby szsori » Wed Jan 27, 2010 10:47 am

Simplicity during the transfer process, first and foremost. All applications would have to do is a fresh pull of their information, as if it were their first time connecting to our site.

emigrating
Site Admin
Posts: 278
Joined: Mon Aug 27, 2007 4:38 pm
Contact:

Re: New site development update - Jan 2010

Postby emigrating » Wed Jan 27, 2010 11:09 am

szsori wrote:Simplicity during the transfer process, first and foremost. All applications would have to do is a fresh pull of their information, as if it were their first time connecting to our site.

Depends on the application :) If the application in question allowed users to change matching criteria manually it may lead to problems. Take Mythbusters as an example, I have every single episode on a harddrive but because TheTVDB listing was "wrong" for such a long time I just re-matched the existing file against what I knew to be the correct season/episode number, now when the epIDs are gone I will have to redo all of this (well, at least theoretically, as the MB listing has now changed it may be it is more in line with my filenames - but it's an example).

Sypher
Posts: 33
Joined: Mon Jun 30, 2008 12:44 pm

Re: New site development update - Jan 2010

Postby Sypher » Wed Jan 27, 2010 12:06 pm

szsori wrote:Simplicity during the transfer process, first and foremost. All applications would have to do is a fresh pull of their information, as if it were their first time connecting to our site.


But that will mean that show id's are changing as well, right? You stated before that they will stay.
Why are they treated differently. Episodes are quite important, without episodes a show has no reason to exist does it?

Most applications link stuff to the TheTVDB Id's, those links will break or the episodes will be seen as "new" which will pollute the systems in question.
For the record, I'm not afraid of chances (most people are) but what I'm afraid of that your new site will force users to change their projects/websites/applications as well.

Isn't there a way to keep the showid's / episodeid's like they are?
I'd imagine everyone would be happier.

If that is not possible, maybe you could consider adding a "legacy-id" field or something like that. This way the transition can be made less painful.

emigrating
Site Admin
Posts: 278
Joined: Mon Aug 27, 2007 4:38 pm
Contact:

Re: New site development update - Jan 2010

Postby emigrating » Wed Jan 27, 2010 2:43 pm

Sypher wrote:But that will mean that show id's are changing as well, right? You stated before that they will stay.

If that is not possible, maybe you could consider adding a "legacy-id" field or something like that. This way the transition can be made less painful.

No, ShowIDs are staying the same. Why Episode ID's were not considered when we discussed this internally I'm not sure...

As for adding 'legacy' IDs I don't think that's a good idea, and I can actually see the reason why both series and episode IDs should be changed - one alternative would be to supply a 'lookup' interface (short-term) where the application in question would update itself the first time it's run against the new API.

szsori
Site Admin
Posts: 1971
Joined: Fri Nov 03, 2006 5:23 pm

Re: New site development update - Jan 2010

Postby szsori » Wed Jan 27, 2010 4:35 pm

Keeping the same episode ids forces us to iterate some of the episode data during the conversion instead of being able to do a bulk select into statement. The result of iteration is that the import takes around 24 hours instead of about 15 minutes, and crushes the server in the process. I can keep episode ids, but it'll absolutely suck every time I need to revise and run the conversion proc. I'm hoping when we near completion of the new site that we can do daily imports and start allowing people to test the new API and site before the old one is taken down. With an import that takes 24 hours, that becomes less useful.

We are storing episode_oldid on the episode records during conversion. One simple method would be to select these out inot the remoteids table so people can do a lookup on them the same way they'd look up a tvrage series id, or a tv.com episode id.