The best way to store site specific config data is probably going to be to create a new component named something such as SiteConfig.cfc with methods such as getProperty(propertyName) and setProperty(propertyName, value). You would then store this CFC in the application scope by doing the following inside Application.cfc's onApplicationStart method like:
<cfset application.siteConfig = createObject("component", "SiteConfig").init() />
Back to your original question though about storing a UDF in the Application scope, below is a way to do that. The basis is that in onApplicationStart you will create a new application persisted struct with your site's config properties like siteName and whatever else. Then a function is stored in a CFM file which is cfincluded only in onApplicationStart, then copied into the application scope. This means that all your regular page CFM files can use application.getProperty(propertyName).
Since the function is only created once and stored in the application scope it satisfies your original question's requirements about "assignment would only happen on application startup".
Hope this helps a bit!
getProperty.function.cfm
<cffunction name="getProperty" output="false">
<cfargument name="propertyName" type="string" required="true" />
<cfreturn application.config[propertyName] />
</cffunction>
Application.cfc
<cffunction name="onApplicationStart" output="false">
<cfset application.config = structNew() />
<cfset application.config.siteName = "My App's Display Name" />
<cfinclude template="getProperty.function.cfm" />
<cfset application.getProperty = variables.getProperty />
</cffunction>
test.cfm
<cfset propertyValue = application.getProperty("siteName") />
<cfdump var="#propertyValue#" />