Umbraco database data dictionary and db improvements Options
LostInSwiss
Posted: Wednesday, April 11, 2007 5:26:34 PM

Rank: Devotee

Joined: 3/31/2007
Posts: 66
Location: Switzerland
The database for umbraco is a core part of the application however there are some design issues that should be addressed.

I can see the umbraco database has been developed by a developer and not a DBA. There is a lack of relationships, selection of the correct data types, constraints, indexes etc. Obviously the unbraco application handles the business logic which is fine and I do not want to get into a debate about where business logic should be enforced but there are some inefficiencies in the current design that are effecting the performance of the front end application and that are impairing scalability.

For example
Data types. We use umbraco stats, the stats table uses a nvarchar 50 data types for storing IP addresses, this is not very efficient (in-fact there are many nvarchar datatapes used throut the umbraco database where other data types could be used. I have seen this befoure developers use a data type they know will work and forget about the consequences). We have 3.5 million entries in our umbraco stats tables and the stats reports take nearly a minute to refresh I am a database guy and I know this should not be the case (100+ mb of IP addresses that need to be filtered by the by possibly has something to do with this!).

Application logic.
I created a bulk user import script for one of our applications. However it appears that there must be a property record for all members of a member type even if that property is null. Not a problem until a member type is updated leading to umbraco crashing when the script is run and a member property is not set. Incedential this is the reason why it is very slow adding new properties to a member type that has many users as Umbraco inserts many many records into the member property table, records that essentially contain no data! This is not a db issue but is related to application logic. If however the db did not allow property data to be null (using a constraint) then the developer would be forced to write their code a little differently. This my be a pain in the ass but would lead to a more stable, efficient and scalable platform.

Whats the problem all this can be fixed in the future
Yes and no. Some schema changes will cause no issues however there are some that would require umbraco core code re-writes. Also there also all those packages and controls that the community has created (Unbraco backup etc) that may run into issues with schema changes.

So what can be done
Is there a data dictionary for the umbraco database available? If so could someone please send me one. I could make a list of suggested changes for improving robustness and scalability though this would have to be done with the active involvement of the Umbraco core team.
hartvig
Posted: Wednesday, April 11, 2007 6:17:46 PM

Rank: Addict

Joined: 3/17/2008
Posts: 953
Location: Nyborg, Denmark
Hi Alec!

You're absolutely right about this. The umbraco runtime runs on cached pre-generated published xml data and as such not really using the db, which is why this hasn't really been a problem performance wise.

I'd really appreciate that you're actively looking into this, and if you e-mail me your CodePlex user name, I'd be happy to give you full access to our Codeplex area (nh AT umbraco . dk). We have a dev forum there and an issue tracker, and it could be cool to have this done for v4. My MSN is support AT umbraco . dk is you have time to discuss this over IM.

Again - your involvement like this is exactly what I hoped for and which is why I made umbraco open source. So much great talents out there, and imagine if we could combine all this!

Thanks!
Niels...

Jeeeez, did I really start this :-)
LostInSwiss
Posted: Thursday, April 12, 2007 1:33:58 AM

Rank: Devotee

Joined: 3/31/2007
Posts: 66
Location: Switzerland
Thanks for the response, I will be happy to help as I am keen to help this project out any way I can. I will mail you my user name.
wboevink
Posted: Thursday, April 12, 2007 8:46:19 AM
Rank: Enthusiast

Joined: 8/11/2006
Posts: 16
Niels Hartvig wrote:

xml data and as such not really using the db, which is why this hasn't really been a problem performance wise.


Niels,

I've to agree with alec, there are no performance issues when displaying the data. But when editing, saving is rather slow and this has to do with correct indexes, relationships and datatypes.

Wouter

kashifkhan
Posted: Wednesday, May 23, 2007 6:14:05 AM
Rank: Newbie

Joined: 5/23/2007
Posts: 3
Dear Alec Griffiths and All,

I have downloaded the umbraco source code (developers edition) and deployed it on my system. For database, I have used the db script bundled in umbraco package for publishing but I have found that there are some db objects missing in the script (e.g. stored procedure named 'umbracoAPIContentGetAll').

Would you please help me in getting the original db script for umbraco source code? Waiting your kind reply ...

Kashif Khan
shur
Posted: Wednesday, May 23, 2007 12:26:45 PM

Rank: Newbie

Joined: 7/27/2006
Posts: 3
Location: Novosibirsk, Russia
Hi everyone,

BTW, Niels, long time ago (2.x related) I've noticed many-many explicit COLLATE clauses in the installation SQL scripts (usage of both Latin1_General and Danish_Norwegian collate sequences in the CREATE TABLE statements).

Maybe it's good time now for DB scripts revision? :-)

WBR, Alexander
hartvig
Posted: Wednesday, May 23, 2007 12:42:52 PM

Rank: Addict

Joined: 3/17/2008
Posts: 953
Location: Nyborg, Denmark
Hi Alex!

How are you?

Yes, that would be lovely. The scripts are old and lacks relations as well. If anyone is interested in giving them a shot, you'll make me a happy man.

/n

Jeeeez, did I really start this :-)
kashifkhan
Posted: Wednesday, May 23, 2007 1:17:12 PM
Rank: Newbie

Joined: 5/23/2007
Posts: 3
Hi guy,

You say right. We should start working on it ... I am with you.

Kashif
LostInSwiss
Posted: Monday, March 10, 2008 9:19:42 AM

Rank: Devotee

Joined: 3/31/2007
Posts: 66
Location: Switzerland
Its been some time but I am keen to get on with this. Anyone interested please write to me alecgriffiths AT gmail dot com
Ruben
Posted: Monday, March 10, 2008 10:08:03 AM

Rank: Aficionado

Joined: 12/21/2007
Posts: 169
Location: Belgium
I'm dropping you an e-mail. :)

Ruben Verborgh | Umbraco Core Team Developer | Working on: Data layer table utility. | LinkedIn
Users browsing this topic
Guest


You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.