|
|
Rank: Newbie
Joined: 1/26/2008
Posts: 8
|
Morning all We are still having serious slowness with our umbraco and cannot determine any obvious issues. The site can be seen at http://realwebsite.punkyduck.net/ sometimes the site is very fast (timeout for viewing) and othertimes it just slows down, even if you viewed the page a few seconds earlier. There does not seem any sql blocking but there maybe some type of leakage as when I recyled the application pool it seemed to respond faster. Information about our build/setup Using build 3270 Running on SQL 2005 Infrastructure is very resilient/multiple servers so hardware is handling a lot bigger applications than little umbraco ! Site as you can see is very simple Does anyone have any ideas/tweaks/suggestions on how we can speed this up before we move the site to another CMS! Thanks
|
|
Rank: Devotee
Joined: 3/31/2007
Posts: 64
Location: Switzerland
|
Looks similar to my (as of yet unresolved) post see: http://forum.umbraco.org/20708#post-20708
|
|
Rank: Newbie
Joined: 1/26/2008
Posts: 8
|
Thanks for your response, will hope someone / somewhere can shed some light. Dont really want to up the threads for the app pool.
Thanks
|
|
Rank: Addict
Joined: 3/17/2008
Posts: 831
Location: Nyborg, Denmark
|
Upping the threads is a bad idea as that making you run in a "webgarden" which is not supported by umbraco (is each thread will have its own cache thus only one cache will be updated on publish). A webgarden shouldn't be confused with a Webfarm (multiple servers) which are supported.
Regarding the issues, I'm very interested in getting more information. I just skimmed through the logs of umbraco.org and the forum and "64-codes" are only given to spiders and crawlers (where it makes complete sense).
Feel free to mail me at NH (a) umbraco . dk
Best,
Niels...
Jeeeez, did I really start this :-)
|
|
Rank: Newbie
Joined: 1/26/2008
Posts: 8
|
Hi Niels
Just came in this morning and the site is timing out again and very unresponsive.
Recycling the App Pool which is dedicated to this app brought it back to life!
Is there anything we can send you, logs etc which can help you determine the issue ?
Or does anyone else have any ideas ?
Thanks
|
|
Rank: Umbracoholic
Joined: 7/20/2006
Posts: 1,069
Location: Charleston, West Virginia, United States
|
Hello, In personal experience, the only thing that seems to make umbraco act unresponsive as you mention is if the processor gets pegged to 100%. If this is occurring, one of the first things I must ask is what types of macros are you using, if any? If you are using user controls, try a few pages that have those controls on them, and watch the system resources. With xslt macros, I have seen some instances with some msxsl:script tags can cause some issues especially with javascript. Also, which version are you using, what is your setup, server config, etc etc. Sometimes a minor detail can point to a separate cause.
• 2007/2008 MVP • 2008/2009 MVP • Core Developer • Certified Professional Level I & II •
|
|
Rank: Newbie
Joined: 1/26/2008
Posts: 8
|
Hi Casey
Thanks for your responses, as mentioned in our first post our setup is
Information about our build/setup
Using build 3270
Running on SQL 2005
Infrastructure is very resilient/multiple servers so hardware is handling a lot bigger applications than little umbraco !
Site as you can see is very simple
Processor is not spiking at all we are less than 5%, there are no SQL locks or long running statements so nothing obvious there.
We use a couple of user controls on a couple of pages but these are very very simple and the slowness affects all pages.
In terms of the macros, every thing is very very simple.
Lastly as the site gets slower gradually to me it seems like some type of leakage, recyling the app pool brings the site back to live at full speed. Just wished it stayed this way.
Was hoping using Umbraco over sitecore would cost us less money!! But if we are spending lots of time trying to resolve issues then in the long run it probably wont.
PS how the tickets selling for code garden! Probably register myself and a developer later today.
Thanks for your help
Shane
|
|
Rank: Addict
Joined: 3/17/2008
Posts: 831
Location: Nyborg, Denmark
|
I can finally confirm this error and I'll spend all resources to find and solve the bug. Shane and others who experience this exact issue, I'd appreciate if you could mail me so we can get rid of it once for all. I'm on nh@umbraco DOT dk.
Regarding CodeGarden move than 30% of the tickets are sold (yay), so I guess (hope) we'll sell all of them during March.
Best,
Niels...
Jeeeez, did I really start this :-)
|
|
Rank: Addict
Joined: 3/17/2008
Posts: 831
Location: Nyborg, Denmark
|
I've also blogged about the issue here: http://umbraco.org/blog/2008/3/5/do-you-experience-random-performance-problems-with-v3xAgain, thanks a lot for brining this to my attention! Cheers, Niels...
Jeeeez, did I really start this :-)
|
|
Rank: Aficionado
Joined: 7/19/2006
Posts: 156
|
It could be a connection pooling problem (website is using to many connections). Adding this to you connectionstring might help Min Pool Size=5;Max Pool Size=500;Connect Timeout=10; Performance monitors are great to find memory leaks or database pooling problems. On the webserver you should add a counter for Process -> virtual bytes -> select app pool Process -> private bytes -> select app pool On the databaseserver you should add a counter for SQL Server General Statistics Object -> User Connections
CPalm, www.cpalm.dk
|
|
Rank: Addict
Joined: 3/17/2008
Posts: 831
Location: Nyborg, Denmark
|
Anyone experiencing this problem could you check if there's a coincidence between the hang and the application restarting? You could do this by comparing hang times with entries in the umbracoLog table.
The connection pool could also be a potential cause.
Thanks for all the feedback, keep it coming and hopefully this issue is gone soon.
/n
Jeeeez, did I really start this :-)
|
|
Rank: Newbie
Joined: 12/12/2007
Posts: 2
Location: Amsterdam
|
Can you check the cmsDocument table? A month ago we expierenced a serious problem.
Every 30 seconds a new entry was inserted in this table. The problem did occur because there occured a bad entry in this table which made the update process keep inserting new verions into this table. This did probably occur because of the fact that the publishing is not into a transaction.
|
|
Rank: Newbie
Joined: 8/21/2007
Posts: 16
|
I emailed, via the contact page, regarding the use of empty catches. For those of you experiencing intermitent, extraordinary slownes, please attach a perf mon and track exceptions per second and total exceptions. If this number is very, very high (we're talking thousands per second or more), we might have to refactor the use of empty catches by preceding them with null checks.
NOTE: Its just an guess and completely not validated at this time. We need to check perf mon to explore this hypothesis.
|
|
Rank: Fanatic
Joined: 11/24/2006
Posts: 322
Location: Stockholm, Sweden
|
I too have had this problem with one of my Umbraco sites. One second I was updating and surfing the public site and then WHAM! - it just stopped responding or loaded the pages so slowly that it was practically the same thing as a dead site. Then I waited for 10-20 minutes and the site was running at normal speed again(?!) The last time it happened I contacted the ISP and asked them to check the server. They reported back that something was overloading the server and that they had fixed it. They didn't say if it was my site or someone else's though... But the wierd thing is that before I moved this site to the ISP web hotel, I had it at my local developing server (Windows 2003). During the final stage of the project my client was logged in to the developing server adding content before the release and almost every time she'd report back that the server just suddenly died or became painstakingly slow and she'd have to give up and go home for the night. Some characteristics of this site: * umbraco v 3.0.2 (Assembly version: 1.0.2774.24063) * Two languages - using hostnames, but only one domain name (i.e. www.domain.com/sv and www.domain.com/en) * The add-ons used: Lefteris' multiple media picker, Doug's XSLT-Search, Flash Image Rotator, Image Gallery. * I estimate that the number of nodes for each language is around 100-150. * No heavy load - it's just a regular corporate site. Let me know if I can help with more info. /Thomas K
Web Developer at Kärnhuset - http://www.karnhuset.net - Stockholm, Sweden
|
|
Rank: Enthusiast
Joined: 8/13/2007
Posts: 33
Location: Sydney, Australia
|
If you have access to your event logs on the server, have a look to see what asp.net errors/warnings have been generated. There are a few bugs in Umbraco that cause the DW20 process to start which will eat up 100% CPU for about 10 mins. I've had some major issues with Umbraco "slowness" but has always come down to some weird Umbraco bug. For instance, if for some reason your IIS user does not have full access to the data folder for creating the umbraco.config file, this will cause a major delay and actually cause your IIS worker process to restart... this is a bug, it shouldn't cause an unhandled exception if the IIS worker process can't access this file. Ensure that you take ownership and replace permissions on this folder to ensure that your IIS user has full access to it. On another note, I've logged a serious bug in CodePlex that basically brings the server to a complete halt and will sometimes kill IIS altogether. I've logged all of the information here: http://www.codeplex.com/umbraco/WorkItem/View.aspx?WorkItemId=14956(It would be good to have both of these fixed) Not sure if you're problem lies within the above bugs. If you can check your processes on your server when your experiencing slowness, this will give you a good idea of your underlying issue (umbraco bug vs poor control/xslt design, etc...) Hope this helps someone.
|
|
Rank: Aficionado
Joined: 7/19/2006
Posts: 156
|
levous wrote:If this number is very, very high (we're talking thousands per second or more Please be aware of "Enable rapid-fail protection" on the Application Pool. Default setting is 5 exception on 5 minutes = disable the application pool. This will give a "Service is unavailable" or "Application restarting". The "Enable rapid-fail protection" could be your problem Thomas
CPalm, www.cpalm.dk
|
|
Rank: Devotee
Joined: 10/30/2007
Posts: 71
Location: Israel
|
Are there any updates, good or bad, regarding this issue?
Itamar.
|
|
Rank: Addict
Joined: 3/17/2008
Posts: 831
Location: Nyborg, Denmark
|
Working on a fix, haven't found the exact issue but a couple of performance issues that could - COULD - be the reasoning. What's really frustrating is that I haven't been able to get a memory debugging dump (via DebugDiag) even though I've been monitoring over ten solutions (on different servers).
However, a 3.0.5 build with the fixes is coming beginning of next week. Hopefully it'll fix the issues.
Best,
Niels...
Jeeeez, did I really start this :-)
|
|
Rank: Newbie
Joined: 3/19/2008
Posts: 14
Location: London
|
Hi all,
Just to provide some more information and background to help diagnose...
changeset: 30642
We've experience similar slow down and narrowed it specifically when calling GetMedia() via XSLT.
The XSLT calls GetMedia() several times (12 times) and we have noticed that the application began to timeout after testing the page a few times and the connection count reached approx 145.
running the sp_who2 command, we could see most connections were to the umbraco database, which tied up with what we saw in performance monitor.
We then recycled the App Pool on the webserver and saw the connections drop. Then testing the page resulted in connections climbing for every page refresh until timeout again when connections reached ~145.
Running the DBCC INPUTBUFFER(@spid) command showed:
select xml from cmsContentXml where nodeID = 1197
And the result was the same for most of the umbraco connections.... this suggests the connection pooling not working? I haven't looked at the code for the app yet because I haven't had time.
We also tried setting the max connection count in the connection string to 250 and then the app is response while we test and refresh until the connection count in the database reaches near 250...
|
|
Rank: Devotee
Joined: 1/10/2008
Posts: 61
Location: London
|
Hi all, I've been working with neilf on this problem and last night we thought we had found a solution. As Neil described it in the post above, we found a problem when using GetMedia several times on the same page. This create multiple DB connections, not being released correctly, and the app pool eventually gets full. I suppose the random behaviour noticed by some of you might be because you don't have a great amount of DB call on a single page, and some connections get eventually killed by the GC. So, we tried the following fix: in the umbraco.DataLayer.SqlHelpers.SqlServer namespace, we replaced: Code:
protected override XmlReader ExecuteXmlReader(string commandText, SqlParameter[] parameters)
{
return SH.ExecuteXmlReader(new SqlConnection(ConnectionString), CommandType.Text, commandText, parameters);
}
by Code:
protected override XmlReader ExecuteXmlReader(string commandText, SqlParameter[] parameters)
{
using (SqlConnection connection = new SqlConnection(ConnectionString))
{
return SH.ExecuteXmlReader(connection, CommandType.Text, commandText, parameters);
}
}
in order to dispose the connection after being used. This fixed our problem with the application pool being timed out because of an increasing number of connections. Unfortunately, that opened another problem. Publishing content doesn't work correctly anymore. Basically if I publish a page, the speech box says it has been saved and published but my umbraco.config file doesn't get regenerated. The only way to get it regenerated it to publish the entire site (from the root Content node). I reversed our hack to the original code (which by the way is changeset 30642), that fixed the publishing problem but obviously not the DB connection one. Haven't investigate yet how the publish process works exactlly but maybe someone can give a hint here ? Maybe worth to add also that the only change I've made to changeset 30642 is the load balancing publishing tip described here. cheers, seb
http://www.be-k.net
|
|
|
Guest |