Load-Balanced servers are not updating on publish Options
drobar
Posted: Wednesday, August 15, 2007 11:20:55 PM

Rank: Umbracoholic

Joined: 9/8/2006
Posts: 1,831
Location: MA, USA
I've just installed umbraco v3.0.2 on two fresh servers, both pointing to the same database. Umbraco runs fine from either server. When I publish, however, the opposite server does not get updated to reflect the change.

At the moment, I have no file synching between servers, but I have only a single DocType, a single Template, and a single Node, so no file changes are being made. I will resolve filesystem synching in the future, but my understanding is that for this simple scenario, everything should be handled within umbraco itself.


I have the following in the /config/umbracoSettings.config file (same settings on both servers)
Code:

  <!-- distributed calls make umbraco use webservices to handle cache refreshing -->
  <distributedCall enable="True">
    <!-- the id of the user who's making the calls -->
    <!-- needed for security, umbraco will automatically look up correct login and passwords -->
    <user>0</user>
    <servers>
      <!-- add ip number or hostname, make sure that it can be reached from all servers -->
      <server>192.168.27.55</server>
      <server>192.168.27.56</server>
    </servers>
  </distributedCall>
  <!-- configuration for webservices -->
  <!-- webservices are disabled by default. Set enable="True" to enable them -->
  <webservices enabled="False">



Does anyone have a suggestion about why this isn't working?

Has anyone gotten distributed calls working in umbraco v3, or should I stick with 2.1.7?

Any suggestions about a good file synching mechanism?


Thanks!
doug.

MVP 2007-2009 - Percipient Studios
drobar
Posted: Wednesday, August 15, 2007 11:50:28 PM

Rank: Umbracoholic

Joined: 9/8/2006
Posts: 1,831
Location: MA, USA
I just installed umbraco 2.1.7, made the same changes to the umbracoSettings.xml file and... it still doesn't publish to the other server.

So the question is... does file synching need to happen (probably for the /data/umbraco.xml file) for every change? If so, what is the purpose of the distributedCall section?

I must be missing something here. Any pointers, anyone?

thanks,
doug.

MVP 2007-2009 - Percipient Studios
drobar
Posted: Thursday, August 16, 2007 4:05:05 PM

Rank: Umbracoholic

Joined: 9/8/2006
Posts: 1,831
Location: MA, USA
It seems that umbraco MUST have at least some file replication/sync between IIS servers, even in the very simple scenario I described. This has to do with alerting umbraco on the other server(s) to refresh their cache.

Niels gave a good description in http://forum.umbraco.org/7478, though that wan't related specifically to load-balanced web farms.

It appears that I'll need to keep the /media/xmlChanged.txt file in sync, at a minimum.


Niels, Sjors, Stevie, and Casey... you guys seem to have solved this issue in the past. How have you done it? Is a SAN the only (expensive!) option?

cheers,
doug.

MVP 2007-2009 - Percipient Studios
chrism
Posted: Thursday, August 16, 2007 5:18:29 PM
Rank: Devotee

Joined: 1/11/2007
Posts: 57
Hi Douglas,
I'm just guessing here, and have no experience with this to backup what I say, but the snippet from the config file you included effectively says that the distributed calls are made by umbraco to call umbraco web services, but then the line at the very bottom says

Code:
<webservices enabled="False"> 


If I were you I would switch this one about and see what happens.

Hope that useful, sorry if its not.

Cheers,
Chris
sjors
Posted: Thursday, August 16, 2007 5:23:07 PM

Rank: Fanatic

Joined: 7/20/2006
Posts: 408
Location: Amsterdam
I recently worked on this, and we found out, that on our production environment only 2 out of 4 servers where updating....

I am not sure what the status is, but i will ask or our application manager did get it working.

Keep you updated.
sjors
Posted: Thursday, August 16, 2007 5:30:11 PM

Rank: Fanatic

Joined: 7/20/2006
Posts: 408
Location: Amsterdam
Ok i did ask, this is what is our situation:

Our client always works on the same machine, the 3 other machines are only used as failover/loadbalance machines)

Actually its little bit more complex, since i wrote a webservice which retrieves information form umbraco by doing a HTTP request, and the webservice makes sure the content is stored in a cache object.

But about Publishing going wrong:

When the publish action is called, there is a call to the second machine in the cluster, to some cachrefresher process function, this function returns a http 500 error. This machine is updated!!!! But then the process stops, so machine 3 and 4 aren't updated.

Maybe Niels can tell what's wrong....
sjors
Posted: Thursday, August 16, 2007 5:30:52 PM

Rank: Fanatic

Joined: 7/20/2006
Posts: 408
Location: Amsterdam
Chris McV wrote:

Hi Douglas,
I'm just guessing here, and have no experience with this to backup what I say, but the snippet from the config file you included effectively says that the distributed calls are made by umbraco to call umbraco web services, but then the line at the very bottom says

Code:
<webservices enabled="False"> 


If I were you I would switch this one about and see what happens.

Hope that useful, sorry if its not.

Cheers,
Chris

As far as i know this is not a 2.17 function, but implemented in V3
drobar
Posted: Thursday, August 16, 2007 9:25:39 PM

Rank: Umbracoholic

Joined: 9/8/2006
Posts: 1,831
Location: MA, USA
I'm going back to v3.0.2 (which is where the code snippet above came from).

I wondered about turning on the webservices... I'll try that too.


Hopefully Niels can give us some insight into these issues.

cheers,
doug.

MVP 2007-2009 - Percipient Studios
drobar
Posted: Thursday, August 16, 2007 10:49:28 PM

Rank: Umbracoholic

Joined: 9/8/2006
Posts: 1,831
Location: MA, USA
Forget that, 2.1.7 is my answer for now. Why? Because I can make it work.

With 3.0.2, even if I copy the xmlChanged.xml file from one server to the other, umbraco doesn't "notice" the change and continues to use its cache. If I touch web.config on the second server then the updated data is used. This happens no matter what I set in web.config and umbracoSettings.config.

Is this a bug in 3.0.2? Has anyone ever gotten it to work? Am I missing something about the way v3 should be set up?

With 2.1.7, I can simply copy the changed files from one server to the other and umbraco instantly notices the change. Easy.

Now, for the truly weird part... on 2.1.7, if I have distributedCall set to True, I need to copy the xmlChanged.xml file to the other server before umbraco will notice the change. That would appear to be acceptable behavior. BUT... if I disable distributedCall, though I have servers listed, the change is registered instantly even without copying the xmlChanged.xml file to the second server.

Here's the code I'm using for this to happen:
Code:

<distributedCall enable="false">
  <user>0</user>
    <servers>
      <server>192.168.27.55</server>
      <server>192.168.27.56</server>
    </servers>
</distributedCall>


I've got identical web.config and umbracoSettings.xml on both servers.

So, is this a bug in 2.1.7, where the distributedCall parameter is inverted, with True being false and False being true? Or have I got a slightly confused configuration after all my trials?


Looking forward to assistance from someone who is familiar with the source code and can give me a hint! Anyone?

cheers,
doug.




MVP 2007-2009 - Percipient Studios
drobar
Posted: Thursday, August 16, 2007 10:54:56 PM

Rank: Umbracoholic

Joined: 9/8/2006
Posts: 1,831
Location: MA, USA
No, wait... after restarting IIS on both servers, the distributedCall setting does need to be set to True... and the changes show up instantly, even without syncing the files.

I'm totally confused at this point, about what does and doesn't work.

MVP 2007-2009 - Percipient Studios
drobar
Posted: Thursday, August 16, 2007 11:12:19 PM

Rank: Umbracoholic

Joined: 9/8/2006
Posts: 1,831
Location: MA, USA
Okay, there was a bit of user error going on in those last two posts :blush:

No question now that distributedCall must be true on both servers. And the xmlChanged.xml file does NOT need to be copied for the other server to be notified of the change.

I only need to sync files when things change in the /css, /media, /xslt, etc. folders.

cheers,
doug.

MVP 2007-2009 - Percipient Studios
hartvig
Posted: Friday, August 17, 2007 9:11:31 AM

Rank: Addict

Joined: 3/17/2008
Posts: 982
Location: Nyborg, Denmark
I'll look into this issue this morning and get back asap.

/n

Jeeeez, did I really start this :-)
hartvig
Posted: Friday, August 17, 2007 10:36:18 AM

Rank: Addict

Joined: 3/17/2008
Posts: 982
Location: Nyborg, Denmark
Ok - I can confirm there is an error (and I've made a fix).

As a part of the publishing refactoring for v3 (for better performance and stability) a whole new set of publish methods was created. The old "umbraco.library.PublishSingleNode" was updated to use this and that method has the responsible of distributing the publish calls.

Unfortunately, the call to this method was replaced in the publishing calls in the UI, with a direct call to the new improved methods. These methods know nothing about distributed calls which is why it fails. No tests revealed this, as they used the call to the PublishSingleNode method (the correct one that knows about distributed calls).

I've made a fix in terms of a new umbraco.dll which can be downloaded here. To use the patch, simply unzip the file and add the new umbraco.dll to the bin (replacing the old one). Naturally, you should *only* use this patch on a v3.0.2 solution.

Please give me some prompt feedback, and then I'll release a v3.0.3.

Cheers,
Niels...

Jeeeez, did I really start this :-)
sjors
Posted: Friday, August 17, 2007 3:39:51 PM

Rank: Fanatic

Joined: 7/20/2006
Posts: 408
Location: Amsterdam
This is great news :) (except investing 15 hours in this ;) )

One thing, download does not works ;(

neehouse
Posted: Friday, August 17, 2007 3:57:26 PM

Rank: Umbracoholic

Joined: 7/20/2006
Posts: 1,076
Location: Charleston, West Virginia, United States
Try here...

The power of deduction...

• 2007/2008 MVP • 2008/2009 MVP • Certified • Licensing • Support • Development • Hosting •
neehouse
Posted: Friday, August 17, 2007 4:01:52 PM

Rank: Umbracoholic

Joined: 7/20/2006
Posts: 1,076
Location: Charleston, West Virginia, United States
helps to actually change the link...
http://umbraco.org/download/umbraco.dll.zip">http://
http://umbraco.org/download/umbraco.dll.zip

• 2007/2008 MVP • 2008/2009 MVP • Certified • Licensing • Support • Development • Hosting •
drobar
Posted: Friday, August 17, 2007 4:08:12 PM

Rank: Umbracoholic

Joined: 9/8/2006
Posts: 1,831
Location: MA, USA
THANK YOU!!

I'll test it this weekend and let you know how it goes.

cheers,
doug.

MVP 2007-2009 - Percipient Studios
neehouse
Posted: Friday, August 17, 2007 4:10:35 PM

Rank: Umbracoholic

Joined: 7/20/2006
Posts: 1,076
Location: Charleston, West Virginia, United States
It worked for me..... :P Thanks Niels..

Side note: How soon will the code be up on codeplex?

Case

• 2007/2008 MVP • 2008/2009 MVP • Certified • Licensing • Support • Development • Hosting •
drobar
Posted: Saturday, August 18, 2007 3:30:08 PM

Rank: Umbracoholic

Joined: 9/8/2006
Posts: 1,831
Location: MA, USA
YES!!

That solved it. Thank you.

For those who want to know, here's what you need to do to make umbraco work in a load-balanced (aka web farm).

1. Install umbraco as normal on the first IIS server, as if it were the only server and not part of a load balanced environment. Run the install script, do everything as you normally would and verify that umbraco is running properly.

2. Modify the /config/umbracoSettings.config file to enable distributed calls, and list all the servers by IP or hostname that are part of the web farm (be sure to include the server you're working on... umbraco won't try to publish to itself, but it needs to be there... see step 3 and its NOTE).

You do NOT need to enable webservices to make this work. Enabling distributed calls will enable the particular service it needs, but this is not tied to the general webservice settings.

For example, the umbracoSettings.config file might contain the following:
Code:

<!-- distributed calls make umbraco use webservices to handle cache refreshing -->
  <distributedCall enable="true">
    <!-- the id of the user who's making the calls -->
    <!-- needed for security, umbraco will automatically look up correct login and passwords -->
    <user>0</user>
    <servers>
      <!-- add ip number or hostname, make sure that it can be reached from all servers -->
      <server>192.168.1.1</server>
      <server>192.168.1.2</server>
      <server>192.168.1.3</server>
      <server>192.168.1.4</server>
    </servers>
  </distributedCall>
  <!-- configuration for webservices -->
  <!-- webservices are disabled by default. Set enable="True" to enable them -->
  <webservices enabled="False">
    <!-- You must set user-rights for each service. Enter the usernames seperated with comma (,) -->
    <documentServiceUsers>admin</documentServiceUsers>'
      <fileServiceUsers>admin</fileServiceUsers>
    <fileServiceFolders>css,xslt</fileServiceFolders>
      <stylesheetServiceUsers>admin</stylesheetServiceUsers>
      <memberServiceUsers>admin</memberServiceUsers>
      <templateServiceUsers>admin</templateServiceUsers>
      <mediaServiceUsers>admin</mediaServiceUsers>
      <maintenanceServiceUsers>admin</maintenanceServiceUsers>
  </webservices>
</settings>





TO SAN, NAS, OR SHARE...
Ideally, you'll be running a SAN mounted as a real drive letter on all of the IIS servers. Thus, each IIS would actually be running against the very same files and changes by any one server would be instantly available to all the others.

STEP 3. Create a new IIS site on each of the remaining servers, and set the root directory to the SAN folder that already contains the umbraco files from your installation on the first server. That's it. Take the afternoon off to celebrate.

This is the best solution. SANs are rather expensive though. Still, if you need load balancing you should consider a SAN as a necessary expense. Otherwise, it might be less expensive to get a single, very powerful IIS server that can handle the high load.



But, assuming you're going to go down the load-balanced web-farm road, yet you do not have a SAN, you will have to add two steps to the above.

Be aware that trying to run IIS sites from a common file share (using a NAS or a file share on one of the servers) is all but impossible with IIS6 and ASP.NET 2.0's security model. It might be possible if the servers share a common domain and user accounts. Even so, it will be a struggle to properly configure the users, share points, application pool users, etc.

Instead, you will probably need to set up some sort of real-time file replication or sync system, either between a master server and all the others in the web farm (a "hub-and-spoke" arrangement), or between all the servers (a "web" or "grid" or "mesh" arrangement).

There are numerous tools available for this, but that's a topic for another post or an umbraco book.


The additional steps are:

Step 4. Having already created an IIS website on each server, copy everything from the first server's website file system. Yes, the web.config, umbracoSettings.config, EVERYTHING. Every server should have an identical IIS and umbraco setup.

Step 5. Set up the file replication/sync system (or confirm the operation of the common share point).




TROUBLE SHOOTING

You'll know everything works because, like magic, what you publish on one site will immediately appear on all the others. When that doesn't happen, there are a few things to check.

If you change the content of an *existing* page in your umbraco web site, publish it, and it does not appear on the other servers, check your umbracoSettings.config file.

If changes to existing content appear, but changes to the CSS, Images, Macros, etc. don't appear on the other servers, check your file replication system.








MVP 2007-2009 - Percipient Studios
seb
Posted: Tuesday, January 15, 2008 12:09:17 PM

Rank: Devotee

Joined: 1/10/2008
Posts: 78
Location: London
Hi there,

thanks doug for the tutorial.
I have followed step by step this implementation, on a small webfarm (1 load-balancer, 2 web server, 1 db server).

I was thinking to manage synchronisation via scripts I wrote (using robocopy); I managed to get a small script that I think does the job. it runs evry minute on one of the server, and files seem correctly replicated.

And still...Publishing content does not update well on both servers.
I guess I would need a bit more understanding on how Umbraco works.
I imagine the data/umbraco.config file is a cached version of the db content. When I look at this file, on both server, they're identicals (and with the latest content published).

Am I missing something ?

Thanks,

Seb


http://www.be-k.net
Users browsing this topic
Guest


You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.