Pictures from DevConnections 2008!

 Click here to see the pictures

 

Building pure ajax applications by Stephen Walther

Building pure ajax applications by Stephen Walther

 

I met Stephen Walther on the shuttle to the hotel.  He was trapped for 20 minutes and had nothing to do but converse with me and a cheerful lady from greece.  There was a good discussion over linq* and some of the new technologies.  I think there are some neat things coming down the pipeline.  I wasn’t able to attend any of his classes except for this one.  Below are my notes.   

 

A couple of recommended tools for benchmarking:

·         http://www.fiddler2.com 

o   how does it work? interrupts response between app and server.  Getting it to work with vista is a pain.  It’s a proxy server, so it works with any browser.  And ie7 doesn’t make a normal request against a local server, so if youa re testing this from your machine put a dot after localhost  http://localhost.page.aspx

·         firebug for firefox browswer.  It is more integrated than fiddler.

 

Benchmark tests against controls:

·         Serverside asp.net: allows browswer compatibility but is bad performance.  All content must be renedered for each interaction.  A simple serverside dropdown list was 2,884 bites.

·         Serverside Ajax hijacks postbacks to make it asynchronous, but still is about 1663 bites for a dropdown list. 

·         Clientside ajax the download dropped down to 776 bites. 

 

Bottom line use strictly javascript and custom paging with a ui control  “Anything else and you are just being lazy”

 

My take: Microsoft is getting deep into javascript and making it more usable.  So I think in  a year we are going to see more controls centered around javascript and client script.  As he mentioned the browser compatibility issues were really big in the 90’s, but not so much anymore. 

 

His blog: http://weblogs.asp.net/StephenWalther/

His company website: www.superexperttraining.com

 

*My conversation with Stephen Walter about Linq:

Basically he really likes it because it’s rapid development.  He thinks it’s an excellent tool and very easy to implement…as well as streamlined.  He admitted he was much more an asp.net guy than a sql guy, but he was pretty excited about it and suggested I give it a try.

I told him of my concerns about overhead (the fact that it seems to pull the entire structure back) and his points were that maybe when you design it , but not when you run it.  It is up to the coder to pick exactly what they want.  He also said that it does some precomplication within the tool to streamline the pull/push of data which makes it faster, and he also said that it doesn’t execute the read of data until you actually pull the data (in other words, in the for loop, not before). 

Anyway, all interesting points.  He said that it wouldn’t be faster than a data reader…and the security implications of putting your sql in your app should be considered. 

 

Microsoft is in a pickle because they tout for so long to always put code in stored procs, and now they are really pushing this.  But overall it depends on what you want to do.  But for the sake of turning applications out quicker it’s a very big step of microsofts to lower development time.

 

 

What’s new in asp.net 3.5 by Steven Smith

The class was a pretty good introduction to the new stuff. Attached are my notes. 

There are new controls (listview, datapager to add paging support, linq datasource (bind to any linq enabled data model…(sql right now, but in the future amazone, flickr, text, xml, etc)

The visual studio enhancements are nice (javascript intellisence, multi-targeting support (asp.net 2.0 or 3.5 your choice), control extender support

Other things in the class: asp.net routing engine (similar to url rewriting), asp.net mvc

Regarding Linq: I asked about the overhead of using linq and he didn’t know.  It’s basically an object relational mapper and a little better in performance than a dataset.  All this translates: I won’t use it.  Currently you can’t use edit/update/delete unless you fetc the entire row. (unless you use an anonymous type…you can get one from www.linqpad.net i guess).  One good thing.  It doesn’t run the query until it needs it (which is why paging support is so much better)

 

Understanding complications, plan reuse and recompilations of stored procedures – Itzik Ben-Gan

Understanding complications, plan reuse and recompilations of stored procedures – Itzik Ben-Gan

 

Can I just say WOW!  The things I learned in this 1 hour class were phenomenal.  Thanks Itzik for giving me a leg up on optimizing queries.  My notes are here (02_understanding-compliationsstoredproc_itzikbengan)  and his sql scripts are here.

 

PLAN RESUSE

When is a plan reused and what is the possible problem?

The problem is that the cost of reusing an inadequate plan is higher than the cost of recompilation

Any query over 1% of an index is not as efficient as a full table scan

How to query the existing plan: select * from sys.syscacheobjects where sql like ‘%something unique about stored proc. He used newid() to get an ide and added it as a commented line in the stored proc%’

·         Usecounts will show how many times the plan has been used

An example would be to create a stored procedure and then run it for the first time with a highly selective criteria.  The plan that is saved might be ineffective if that’s not a typical criteria. 

Possible solutions:

Create an index that covers the ctriteria(n))

Note: when it doesn’t matter the selectivity it is called a trivial plan and the same plan is always usedc (the optimizer has a flag for this type of plan)

Add recompile to procedure

Specify certain queries not to optimize (add option (recompile) after where clause

The cost of recompile every time is less than the cost of using a wrong plan

 

PARAMETER SNIFFING

The problem is centered around the fact that a declaration inside a stored procedure cannot be ‘sniffed’ (read ahead), therefore it cannot be optimized.  Great discussion on this read notes for solution

 

DYNAMIC FILTERS

This was another great example that showed a common <bad> use of dynamic sql. (example….where name like ‘%@parameter@%’ or  city like ‘%@city%’ or state like ‘%@state%’)  He talked about a way to optimize a query in a clever way.  He also spoke of how to prevent sql injection (use sp_executesql with parameters).  Excellent information I can use today.

Query Programming and Tuning - Itzik Ben-Gan

Query Programming and Tuning – Itzik Ben-Gan

 

I can’t express enough how much fun it was to take Itzik’s class.  It was my first class with him, and now I know why he has such a huge following.  He has a grasp for understanding the structure of the query language and a passion for database logic.  I took a 6 hour class and have attempted to encapsulate my learnings in 25 pages of notes here.  Also, you can find the sql for this class on the references section of his blog (www.insidetsql.com) or by clicking here.

 

Highlights:

THE OVER CLAUSE

“The over clause is the single most powerful thing in sql language”  because it bridges the gap between iterative logic and the logic of sets.  It is optimized well, orders well, and has a group of ranking functions.  The different ranking functions are covered in good detail.

DATA AGGREGATION TECHNIQUES

Subqueries: expensive.  Using subqueries is a separate scan for each subquery.  Multiple joins are better than subqueries as they do one scan fo all.

Cursors: sometimes you have to use (benefits: can control order of elements and doesn’t impose restrictions)  However the cost is extensive.  It is several times slower than a basic aggregate

Grouping:  single pass through data.  This is a good solution unless you need a running aggregate.  However, the over clause will give us better performance for this case.

Sliding aggregates: similar to grouping but we fixa  a low point and an upper point (1 month ago to 1 month ahead)

Custom Aggregates: most common forms: string concatenation, median, mode

Pivot: Using pivot is useful but it’s not any better performance than grouping…it isn’t useful if there are large elements in the group….and it’s not straitforward (for someone else to administer later)

CLR User defined aggregates: 2005 is restricted to only 1 aggregate function with code and are restricted to 8000 bytes.  Using clrs is the fastes way that he knows to do sting concatenation, but it won’t be easy to maintain.

AN AUXILILARY TABLE OF NUMBERS (what is it? a single column with a sequence of integers)

TIP: in 2005 you can use a synonym to make it look like a local object. 

Use this table To SPLIT ARRAYS  (the best way to split an array as far as performance is to use a clr).  This is a great section that you should be sure to read as EVERY organization finds a need for this, and most of the time we end up writing functions that use poorly optimized resultsets. 

TOP

The problem with top is how it is used: Select top 3 * from sales order by qty     Here TOP needs to use ORDER BY  for filtering purposes (which 3 rows to pick).  The order by clause now has two meanings: display and filtering.  There is no guarantee of presentation of the data.  Instead use the OVER clause.  (lots of examples in notes)

T-SQL VS CLR

If it’s data manipulation, use t-sql

If it’s iterative logic, algorithmic logic,k computational intensive clculationrs, string manipulation, parsing then use .net. 

 

Ado.net Performance tips and tricks by Don Keily

I attended a class by Don Keily. 

It was a good class, confirming a lot of what I felt to be true, reminding me of some things, as well as teaching me several things.  My notes are here: 03_adonetperformancetipstricks  but I put below what I got out of the session.

Ado.net performance tips and tricks by Don Kelly

 

·         From a performance standpoint linq to sql is evil

·         Don’t do data paging on the page. Do it on the server

·         Change the form of data rarely, if ever

·         Don’t let ado.net let you be lazy

·         Command builder causes several trips to the server

·         The more tiers the slower the performance

·         Avoid mapping object to tables and columns

·         Don’t build deep object hierarchy unless you need it (don’t create a customer object if all you need to do is modify their name)

·         Do check out performance tag on ado.net team blog (http://blogs.msdn.com/adonet/default.aspx  )

o   http://blogs.msdn.com/adonet/archive/2008/03/27/ado-net-entity-framework-performance-comparison.aspx

o   Shows you a chart…a breakout of where ado.net and the entity framework are different and the cost of the different calls

·         Do use data caching at every opportunity

o   Application

o   User specific

o   Keep track of update frequency and update caches differently based on need

o   Update it as close to the application as possible

o   Minimize transformations

o   Expire it sanely

·         Be aware of the cost of moving data between layers.  And know your options

o   Dataset

§  Lots of features and bloat

o   Typed dataset

§  More of a programmer convenience

o   Datareader

§  fast

o   Xml

§  Bloat.  Text and bloated.

§  Not the most performance way to move data

o   Custom class

§  Define own class carefully design the serialization so it goes into a small footprint.

§  You only want to serialize the dat athat is relevant

·         Reduce round trips (make batch sql statements with 2 statements in one)

·         Use connection pooling.  (no need to fine tune, just use it)

·         Don’t use builder objects

·         Set nocount on (stops dbs from sending the number of rows that were affected)

·         Close things and don’t wait for the garbage collector to do it (why? If you don’t garbage collector has to make two passes)

·         Use the USING keyword (using (sqlconn conn = new conn)………do stuff

·         Don’t have to explicitly OPEN or FILL data adaptor.  Just use da.fill(dataset) and you don’t have to open or close connection

·         Don’t have to explicitly open for update method

·         Let ADO manage connection pool (share connections, avoid per-user logons, don’t pass connections around)

·         Use paging liberally

·         Use executenonquery over executescalar (former doesn’t build a recordset)

·         For wide rows or blobs use commandbehavior.sequentialaccess.  It works like a datareader for columns (once you get the 5^th field you can’t get previous)

·         Never build a commandbuilder at runtime

 

SSRS 2005 Advance reporting design

SSRS 2005 Advanced Reporting Design

I took a class by Jason Carlson on Advanced reporting design.  I’ve been working with Report Services for a while now, so honestly there wasn’t much for me in the class.  However, I did learn a few tricks.

1.  Filter all reports on page with a list.  This feature also enables group filtering and group behavior for whatever you put in the group (using list control)

·         Add list from toolbox and use it. cover chart and table with list (make sure it’s embedded)  they will now operate exactly the same.

·         Move the filter from the list to the data region (list properties

·         You have to have a group in the list.  So give it a fake group

o   Use Expression  : 1=1

2,  You can add an assembly to your site and access it from your code.  More on this later but the basic functionality is Code.object.functionname

3.  Inline bar chart tip (using a bar chart in a table).  Very clever!

 Use padding to size the line.  Use a formula:

=((1.0-Fields!SaleAmount.Value/Max(Fields!SaleAmount.Value,”TopOrders”))*72) & “pt“

 

(Percentage of total 1- (value / Max(value)) Multiply by 72 pts in an inch, the column is 1 inch wide)

New Features of SSRS 2008

I attended the class New Sq Server Reporting Features 2008 by  Jason Carlson. 

The highlights are below.  The full notes I have are here.  Due to legal constraints I’m not sure that posting the slides would be ok, but I’m leaving the references.

1.       Not hosted in IIS any more  (however it does use http.sys which means it can communicate with IIS).  It uses sql server networking stack instead. (?) There were many settings that they discovered that conflicted with report engine.  See slide on ssrs 2005 vs ssrs 2008 (page 6) to see architecture differences

a.       Settings not supported

                                                               i.      Isapi filter

                                                             ii.      Anonymous authentication

                                                            iii.      Digest authentication

                                                           iv.      Client certificates

b.      You can now pick your own root, not IIS’ root (graphs!)

2.       Memory managmement

a.       Dynamic self-monitoring under memory pressure

b.      Uses file system cache to adapt to memory pressure (receives memory events from server)

c.       Administer can set targets

3.       Suggested using sharepoint with it.

4.       Reports are now hierarchial cursor-based object model

a.       Rdl is exposed as an object model

                                                               i.      Object model reloads based on which group is shown (?)

5.       Report engine modifications

a.       Report processing is not on-demand (only grabs first page.  Expressions will never run until you get to that page)

b.      New rendering architecture

                                                               i.      Slide 17 shows architecture

                                                             ii.      They wrote all their own renderers

                                                            iii.      New word rendorer

6.       Report builder modifications

a.       Format updates to text, paragraphs, lists, etc.

b.      Can use [fieldname].  No need for “this text “ & field.bla.value & “ results in whatever”

c.       Tablix (multiple parallel row/column members at each level)

                                                               i.      Slides 28-33

7.       Other

a.       Still no css, but it is the number one request they receive

b.      Gauges can now be dropped into reportview and used and every series can have it’s own report type.

 

New features of IIS 7

I attended a class on the new features of IIS 7 taught by Carlos Aguilar Mares.  It was a great discussion and looks promising.  I’m looking forward to a more simple IIS 7 framework.  The common scenario that I see is that most don’t know anything about IIS (myself included), so I’m hoping some of the architectual changes will help with that problem (an example: what does application pooling do exactly and how do you set it appropriately?) Anyway, here are my notes (02_newiis7features).  And Be sure to check out Carlos’ Blog. 

Carlos Ag’s blog

 

Carlos’ blog on this topic 

 I wanted to mention, as an aside that some of the command line features of iis7 look pretty good.  For instance,

Appcmd list  will list all the site objects

Appcmd list requests will list the currently executing http requests

Appcmd list config “defaultwebsite/temp”   this will show you the web config, but all the inherited default documents as well!

 

Here’s a nice page I found regarding appcmd.  http://learn.iis.net/page.aspx/114/getting-started-with-appcmdexe/

 

Also, the applicationhost.config supports “%windir%” and %systemdrive% arguments when pointing to modules, virtual directories, etc. 

The Case Against the Batch

The next time someone asks me to write a batch file I’m going to say no.  I end up having to write one every 5 or 6 months and they always give me problems, whether security or switches or a misunderstanding on a command.  This one creates a backup directory, and then copies files over from a source location to a destination.  It sounds easy, but the reality is that this simple request has moved from ‘just copy files’ to ‘copy files, create a full log to show errors, make a backup of all existing files, and let my assistant be able to run it.’  Soon I expect them to ask me to enable it to only copy certain folders and make it shoot fireworks when it’s done.  The point is: for production don’t use batch.  Use a vb app

• Catching errors isn’t easy..it involves piping out all your cammands in hope that one of them contains an error which they usually don’t. (it seems like there is a way to catch system errors but i can’t find it) 
• VB app has error handling
• VB app has debugging
• Sending emails is easier (i know there are handy dandy tools but there is a comfort level difference here that makes my point for me)
• Security: anyone with access to the bat file can modify it
• Extensibility is easier
• It doesn’t feel like I’m rewriting the wheel. 

There is a bottom line here: program in a technology that you are used to or want to.  Your case can be made simply under the notion that it’s what you know and are motivated to do.  If you don’t know it or aren’t motivated it will take you longer.  Stand up for your self.  With that said, here’s my batch file that runs every day that I wrote because I was asked to and I didn’t talk back.  : )

echo on
REM For testing purposes my production destination is c:\temp
REM this batch file copies existing files to a directory dated today, deletes the existing files, and copies from a source location

REM STEP 1 check for flag before copying
if not exist \\sourcelocation\c$\website\Content\CopyContentYes.txt goto end

REM STEP 2 CREATE DIRECTORY WITH DATE
Set mm=%DATE:~4,2%
Set dd=%DATE:~7,2%
Set yyyy=%DATE:~10,4%

REM if backup directory exists then GO TO delete files line (else create directory and copy files)
if exist c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd% GOTO DELETEFILES
mkdir c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd%

echo STEP 3 COPY LIVE CONTENT FILES TO BACKUP FOLDER >c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd%\copylog.txt
xcopy c:\temp\content  c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd% /y /e /c >>c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd%\copylog.txt

echo STEP 4 DELETE FILES in content files ON PRODUCTION SERVER >>c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd%\copylog.txt
:DELETEFILES
del c:\temp\content\* /q /s >>c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd%\copylog.txt

echo step 5 show all the files in the source that are about to be copied
dir \\sourcelocation\c$\website\content /s >>c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd%\copylog.txt

echo STEP 6 COPY FILES TO NEW DIRECTORY >>c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd%\copylog.txt
xcopy \\sourcelocation\c$\website\Content c:\temp\content\ /y /e /c >>c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd%\copylog.txt
echo STEP 7 DELETE COPYCONTENTYES.TXT FILE IN LIVE DIRECTORY >>c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd%\copylog.txt
del c:\temp\content\CopyContentYes.txt >>c:\temp\contentBackup\ContentBackup_%yyyy%_%mm%_%dd%\copylog.txt

:end

Things I learned while creating this

•  Del command with any of the /a switches will only delete files with those attributes (not including those files)
• > will pipe out your results to a log
• >> will append to an existing log
• How to create a directory with using a date (this script creates a directory called ContentBackup_2008_04_0
• del /q suppresses confirmations and xcopy /q is quite mode (doesn’t show the files being copied)