You, Ben W and Dagobert:
OpenCSW Wintercamp in Munich, Day 1

2010-01-23 minutes, Munich

Yearly board vote

dam: No e-mail to the board can get lost.
william: Debian allows private voting, with signatures.
dam: Is it open source:
william: No idea.  It's complicated.  Is it worth doing?
skayser: Is board a mailing list?
dam: I think it's a mailing list, but it's not public. I don't know the details. It was set up a long time ago.
w: If we want to have a private vote, maybe not this year, but maybe we can develop a small website, people would log in with a certificate, we could check their identity and then check their data.
maciej: The web server would have the logs.
w: Not necessarily. If you use SSL, you'll have the log that someone was there, but not the decision.

11:17 voting on the topics

11:23

dam: We should reactivate the buildbot.  Package version bumps should be done automatically.  Update the GARVERSION, and buildbot should build the new one, and put it into testing. As a maintainer, I should only check if the package works.  Tedious routine tasks would be reduced.
Jan 23

Trygve:
I wonder if the effort is worth the value gained from this. It is not editing the Makefile that's the part of the process that take the most time, actually building the package is where the most of the time is spent.

I'm not apposed to it at all, it is valuable. Just wondering if there are other efforts where we could gain more with less effort.

One place where it would be *very* useful is if we want to make sure that recent changes to GAR didn't break anything and we do complete rebuilds of the entire stack. That would validate lots of stuff, from GAR to compilers, download URLs and everything.


b: Any reason to abandon this?
maciej: I was busy with other things.
dam: We didn't have gmake platforms at the time. We have them now. Upstream watch is now enhanced.
w: There are problems with uw. Sourceforge pages have changed. UW doesn't work with SF pages. I have to check this. I'm rewriting UW to use a different way to check upstream and work with the pkg db. I wasn't sure if it's worth to quick fix the old version.  It's a simple script, adding complexity is not easy. Maybe it's worth doing, I don't know. I have to look deeper. I don't know when the next version is available. There are also few problems with some websites.  I can deal with 2 kinds of downloads from upstream websites: the generic way, where I can access a page with the link to sources. I'm parsing the page with regexes and I look for the new version.  SF is a special case.  The way you browse SF is different, more parsing is required. There are some websites which are all different. Some other websites, there's only one link with no version information. It's unfortunately about half the projects in gar.
skayser: They simplified the downloads on SF.
w: The problem I have is that for several projects, the way UW is parsing the page, it's difficult to parse only a subsection.
maciej: Any parsing library?
w: UW would have to be rewritten. I can do it but I was rewriting it anyway with the pkg db. But maybe it's worth fixing the issue with the old one first.

Automatic builds

maciej: People didn't want buildbot because of resource usage.
dam: install it, if there are problem, reduce it. I can even do it right from the start. Patchclass resource control.
...
maciej: People OK with adding categories?
skayser: You now have to remember the categories.
dam: Just use "default".
w: For people who use completion, it's better to have categories. Autocompletions is slow with dirs with lots of subdirectories.
dam: grepping would be more consistent with categories. Ironically we moved away from this.
skayser: Can we get rid of the semantic overlap between the two 'categories'? Rename to 'builddefaults'?
dam: We have more stuff to rename. Maybe do them at the same time?
skayser: What's so hard about renaming? We can use the FlagDay.
dam: There are announcement of major changes. If we have an old build recipe, and it breaks, you can look there. I don't know if we should adjust everything. We should focus on changing everything.
maciej: You can write a script to change gar and builds at the same time.
dam: OK.
skayser: We need an standard announcement, so that people know that there's a large change coming in and they update their sources.

william: When I'm parsing output from GAR I can not retrieve info about the maintainer.  There inconsistencies between the garname and package name and the name in the path.
dam: It would be solved by pkg db, or a specific target which would list package name, or solved by functional e-mail addresses.
william: functional e-mail adresses are very useful, because they make it easy to communicate with users.
dam: Would be good to have all info about packages on the web just like Debian has it. www.opencsw.org/packages/pkgname with all the previous versions and a link to a wiki page that the maintainer can edit. All the versions and all the bugs like we have now, but more info. We would need pkg versions in mantis.
skayser: I did something for versions to show up. Now that we have the web dev zone, I can experiment with mantis in there. A dropdown with all the versions.
maciej: Do we have a consensus about the categories?
all: Not really.
benny: We could use gmake search name=lsof which would search inside the categories.
dam: We would also like to change PREREQUISITE_PKGS, because they are similar in structure.
maciej: PREREQUISITE_PKGS => BUILD_DEPS and REQUIRED_PKGS => RUNTIME_DEPS
dam: I don't like the name. GAR also has the concept of dependencies, with directory dependencies. But the dependency mechanism is out of order. I think that it was used some time ago. DEPENDS, LIBDEPS, BUILDDEPS - this should be stripped completely. BUILD_DEP_PKGS, RUNTIME_DEP_PKGS would be better.
maciej: OK
dam: We should strip DEPENDS, LIBDEPS and BUILDDEPS from the GAR code. They are not used any more. It can be then used to "gmake world". This would be harder if not everything is in GAR.
maciej: GAR can call out to other build system if the build descriptions are in the repository
dam: There are inconsistencies in the naming, should we rename them?
skayser: people who are developing will be watching the announcements
dam: We need to update the external references, where we reference the GAR revision, we should change to gar at specific rev.
maciej: Why not update all the builds at the same time?
skayser: I think it's easy.
dam: hm... ok, it's ok. We should change the tags, to reference GAR at the time when the build worked. trunk would reference mgar/v2, and tags would reference mgar/v2@<revnumber> so the tag keeps working even when gar changes later on.

dam: so, buildfarm.  The new website puts GAR forward as standard.
skayser: What's the status of the new maintainer welcome page? <looks at the current maintainer welcome page on www.opencsw.org>
maciej: MUST RUN AROUND SCREAMING!!!111oneoneone. 
dam: We should correct it on the new page.
dam: The package page should have a link to the build recipe.
all: +1
skayser: dam, Nice work on getting new upstream accounts for non-maintainers!
dam: Yes :-) It's on the website, http://www-mockup.opencsw.org/extend-it/signup/to-upstream-maintainers/
skayser: We can move it one level up in the menu
dam: It's cool that it's now part of the open source ecosystem.

skayser: new item, update the classutils wiki page
dam: we haven't accomplished anything, because there was a warning and Phil didn't like it.
skayser: why don't we suppress the warning? but Phil says that if you suppress them, you're not security conscious. The solution we came up was to move scripts to /opt/... and then create the stubs into the package, like i.foo, which would be only stubs.
dam: But the pkgadd displays a security warning if the class action script is outside /usr (in the package or in /opt). Maybe we should write up why are we not sparse zone friendly at the moment. Then it's not forgotten. In a year we'll still remember it.
skayser: I can do that.

bwalton says: do we have to support _all_ types of solaris installs?  we're jumping through hoops for nfs /opt/csw already.  i suspect sparse zones are more common than that, but still.  is there a line that can be drawn?
Jan 23

Trygve:
I wonder if sparse zones are important. they're going away with opensolaris/solaris 11 it seems.
Jan 23

You:
I would be personally interested in having them supported, because I have lots of sparse zone installations at work.  I don't know how they are generally popular.
Jan 23

Trygve:
Yes, I think it is very valuable supporting it. They seem quite popular to me, I think lots of people find them useful. ( I know I do)


maciej: how about a break?
all: ok, let's have lunch

maciej: (reads bwalton's question)
dam: Good question.
maciej: What kinds of installs are there?
dam: 
NFS-Shared /opt/csw for homogenous Solaris released
NOT supported: NFS-Shared to different Solaris-releases (especially mixing Solaris 10 and an older release)
Unsolved problems with shared NFS
Config files
Start scripts, both SMF and /etc/rc.*/*
Full-Root Zones
Sparse Zones
with private /opt
with shared /opt

(coffee being prepared)

maciej: Let's talk about the website.
skayser: (projects the wiki page http://wiki.opencsw.org/new-website)
maciej: Let's go through the TODOs. Let's have 3 stages: 1. absolutely necessary, 2. after the switch, put out the fires, 3. improve stuff. Do we have to verify the internal links?
dam: There are tools to do that. I don't remember names though. They report any broken links.
Jan 23

Trygve:
Are there that many pages that we actually need a tool for it? I remember there being like ~20 PHP pages.

Even Phil won't be able to stop that just because of a couple of broken links.
Jan 23

You:
I would classify this in the 2nd category: do it after the switch.
Jan 23

Trygve:
+1 (damn Google for hating threads)

skayser: What if there are fully qualified links to ww-mockup?
maciej: They will still work.
william: web-httrack can verify a website
maciej: Next item, verify that the current links are preserved.
william: Pretty much the same task, you can ask the software to follow the external links.
maciej: Manual checking would be sufficient. There's /packages /maintainers /search /userguide /mirrors /standards. Later on we can develop a tool. I have some code already. For the switch, manual checking only.
maciej: The permalinks as described on the wiki. I  would stick with opencsw.org/singleword.
dam: Should we confine packages to a specific subdomain. pkgs.opencsw.org for example.
Jan 23

Trygve:
dam: might be nice as it's easy to deploy separate applications on each domain, but that's for later IMO.

maciej: I don't see why not, but would need to provide redirects.
skayser: It would be a URL to view packages.
benny: Debian has domain/release/pkgname
Jan 23

Trygve:
benny: We don't have released as it is now. Only current.

maciej: What could we have then? pkgs.opencsw.org/current/bash?
william: If we choose to use the release name in the URL we'll only be exporting information about the current
maciej: We can provide information about all the releases testing/current/stable on one page
william: If we want, we can append the release name to the url: pkgs.opencsw.org/bash/stable
Jan 23

Trygve:
All we have to do to have "releases" is to just use a date as a release name (similar to what opensolaris do)
Jan 23

Trygve:
Having proper releases would solve a *lot* of problems for us as it is the basis for being able to change policies and migrate.

Now all migration talks are stopped because "we can't break stuff". But if we require people to upgrade from one release to another we can break stuff softly and require people not to jump releases.
Jan 23

Trygve:
After the last meeting I started writing software to handle massive amounts of repositories, both for "release" repositories but also for team and experimental repositories. I want to continue on that work if having releses is something the group see as valuable.
Jan 23

You:
I like this idea, keep doing that. We have releases as one of the later points on our agenda.
(groups switching to a different internet connection)

all: yey! It's faster!
william: Does the search and maintainers have to be done before the switch?
Jan 23

Trygve:
I would say yes too as it's an important part of the site and it's already there. Just copy the PHP pages.

Integrating the L&F can be done later.

maciej: I say yes.
maciej: Do we want the latest releases as category 1?
Jan 23

Trygve:
I'd say (3), and hopefully they can be autogenerated from either the database or Phil's email.

skayser: (shows his wordpress stuff on www-mockup) We could have the last packages like the fink project does.
maciej: How hard is it to do?
skayser: There's no DB. Phil has a script right now.
maciej: Can we reuse the same file that he's already generating?
skayser: It's an RSS feed, so I suppose yes.
william: What we have now is a complete feed, it has each and every package that has been updated. Maybe it doesn't have to be on the front page. There are 6 important news. If you add releases that can happen on every day, we can release 50 packages, and the important announements would be flushed out. Maybe we can have only the announcements in the main feed, and a separate RSS with the package.
Jan 23

Trygve:
I think it's important to think about what which consumers of the data we have.

I kinda doubt the users would like to know of *every* release that's done.

dam: I would like to show a RSS feed that is very frequently updated.
william: yes, but they have to be separated. there can be a list of all the feeds that we're providing. One with important announcements that is not very frequently updated, with higher level information.
skayser: Imagine you're coming to this place for the first time (shows website) these dates are old, it's November 2009, it's old, looks dead. People land on the page and.... (shows fink) I've never clicked on them, but I see that there is something happening
Jan 23

Trygve:
How about a "package statistics" box on the front page? 

Can have stuff like:

* Total number of packages
* Packages released last week/month
* # of Contributors
* Last stable release

william: Maybe another section, with all the packages in a separate window, a different level of information. One would show project activity, and the other would have more written information.
maciej (reads Trygve's text aloud)
all: we would like that
william: That's easy
maciej: Which stage?
Jan 23

Trygve:
Definitely later as Dago said.

But the point is to not clutter the start page too much, it's important to know our customers.

dam: Later, later. (3)
maciej: To recap, recent packages are stage 3, agreed?
Jan 23

Trygve:
I would suggest that we show the 3-5 last packages released at any time with timestamps just to show activity. Rendered in a box, not as a news item.

Major releases/important releases go as news items.

dam: For now, we can have a few later package
maciej: And that's stage 1?
dam: yes, it's easy
maciej: Who'll do it?
william: I will.
skayser: I just found the scripts from Phil
maciej: It's short, we can steal his code and reimplement it in PHP
william: is it RSS or HTML?
maciej: HTML, but there is more than 1 script.
maciej: http://www-mockup.opencsw.org/get-it/releases/ - check wording?
bwalton: LOOKS GOOD 20100129
dam: delay it until we talk about releases?
maciej: Let's assing priority. I say 3.
dam: OK.
maciej:  http://www-mockup.opencsw.org/about/history/ should be level 3
william: I say priority 1. Say how and why OpenCSW was founded, and not talk about the fork.
maciej: Who will do it?
william: A native speaker?
skayser: This is sensitive topic, and he wrote those pages. Someone fluent in English needs to write it, and we need to review it.
william: It will be read by people like Dennis, we have to take care of the wording.
maciej: Shall we ask Ben?
william: I think so. Maybe Ben can also talk with Dago about the content.
dam: I guess it would be best to move the the first paragraph from the history page to the about page. 
william: When the new website is up, Dennis will read every word carefully.
maciej: Which paragraphs, Dago?
dam: I would move content from the history page, from the top to "...and domain name." to the about page, and leave the rest on the history page.
maciej: What would Ben do?
dam: Nothing.
maciej: OK
maciej: " Check pages and consistently place code tags around inline file names ", I say level 3
skayser: I've done most of this already.
maciej: " Verify that pages are install tool neutral "
dam: Peter is the one who cares, he can do it anytime he wants.
maciej: Level 3 then.
skayer: I've never seen any other project advertising 2 package installation utils. Debian switched to aptitude. The don't say use this or the other.
dam: Sensitive topic.
skayer: Yes, but our pages say: "you can use either this or that"
amaier: They will pick the first one. The position has to be changed.
maciej: Should we remove info about pkg-get completely then? Or have a "legacy" page?
skayser: I say drop pkg-get from the menu. But we would have an endless fight about everything.
Jan 23

Trygve:
pkgutil vs pkg-get should be confined to a single "getting started" or "installation" page.
Jan 30

Ben W:
Personally, I'm in favour of dropping pkg-get completely...but, if it's to be kept, I agree with Trygvis here.  Mention it only once, and behind pkgutil as people will use the first tool suggested.

amaier: Is it worth it?
skayser: Maybe add a comment: manual mentioning both tools will bo confusing.
dam: (changing topic) why do we have "get it" and "use it". What's the difference.
william: Get it is installation, Use it about configuration.
dam: I see. What's behind it now? (looks at the page)
skayser: See the pkg-get and pkgutil? It's repeated everywhere.
maciej: I agree with Trygve, we can have a single page with pkg-get and pkgutil, but everytime we give an example, we give pkgutil.
Jan 23

Trygve:
I think that might start a war or two. How about just using the phrase "install package foo" instead? I think we can assume that they know the basics on how to use pkg-get and pkgutil.

dam: Having both examples doesn't hurt.
maciej: I think it does. It's confusing for newcomers, and you don't want them to be confused.
dam: What about a package installation cheatsheet? We can have one page explaining both. 
Jan 23

Trygve:
Dam: Is it required if there is an "getting started" page?
Jan 23

You:
It would be more extensive comparison of both tools. Can do overlays yes/no, etc.
Jan 23

Trygve:
Ok, that might be useful as long as it's confined withing a single page.
dam: When I look at a website and look for info, I just look for copy/paste code for instant use.
Jan 23

Trygve:
For specific news items where you're talking about the latest stuff I wouldn't mind if pkgutil was used. We're talking mostly about the "static" pages how, right?
Jan 23

You:
Yes, it's about the static pages. I'm all for having copy/paste stuff, and that would use pkgutil only.
dam: On the package page for each package /packages/<pkg> there should be something like "First time user? On Solaris 10 use pkgadd -d ... && pkgutil -i ... or click here for an introduction"
Jan 23

Trygve:
I honestly don't see the value in that. We can assume that users have certain knowledge on how to use software in general (like make sure you at least sweep the getting started stuff).
Jan 23

Dagobert:
Please remember that the specific package pages may be linked from the upstream pages as sources for binary packages and that those users may not know OpenCSW at all.
Jan 23

Trygve:
Still don't see the value really, they have to start using OpenCSW first. The package name should be enough for everybody.
Jan 23

Dagobert:
When I come to a Debian package offering direct download I find this very useful. Just MHO
Jan 23

Trygve:
A direct download link to the package is useful, but I see this as basic information and thus cluttering to have it on all the packages
Jan 23

Dagobert:
Agreed. But we should have "First time user? Please read here how OpenCSW works" maybe as footer.
Jan 23

Trygve:
We're getting very specific, but I think a first time user would go to the front page to get the basic information.
Jan 23

Dagobert:
We could look at the referrer :-)
Jan 23

Trygve:
Clever!


dam: More work on this TBD on the package summary page scribble up in the afternoon

maciej: To recap: cheatsheet with the comparison, pkgutil in the examples.
skayser: Who'll do it?
maciej: I say the cheatsheet is Peter Bonivart.
dam: He'll (maciej invites skayser to the wave) love to write this page.
maciej: And who'll do the code examples? Also Peter? He's already declared that?
all: mhm.
maciej: Which priority? 3?
all: mhm

all: (getting distracted)

maciej: I'll try to update the wiki now. (updates the wiki) Done.
skayser: william, how did the integration with wordpress go?
william: I got the packages page to work out of the box. But I have to modify the page to achieve the same look as the rest of wordpress pages.
benny: Do we want to have the same subdirectories in wordpress?
william: We'll preserve the URLs. In some cases we might to HTTP redirect.
maciej: william, all 3 priority 1 itemas are yours.
william: I already have some code, it's 2-4h of work. You'll then verify the links. For starters, I'll leave some statistics out.
Jan 23

Trygve:
I would put those two integration points as "to be done later" 

maciej: looks good to me.

all: (eat cake)

(discussion about the logo)
benny: If people see a different logo 2 years later, they won't recongnize it.
dam: Let's add it as the todos.
maciej: Which level?
dam: 2 or 3.
skayser: Anybody knows a designer?
dam: I like the design where C is part of the O.
skayser: The sausage?
dam: Yes! :-)
skayser: Maciej said that 'blastwave' is catchy.  It would be cool to have something eye-catchy.
william: They have a logo?
skayser: They have something
william: Nice picture, but...
maciej: Dago, do you have vectors for the sausage?
dam: Yes, hang on... I have an EPS. I'm pretty sure she also has it in illustrator.
maciej: I use inkscape. I could try fiddling with it. I think the sausage can be improved.
maciej: (processes design and technical points)
maciej: william, look at prio 1 stuff, can other people help you?
william: If I have the code to generate the latest updates, it's easy. I just need the root access to the web dev zone. After that it's 1 day of work, apart from integrating the maintainers and search scripts, which should take about a week.
Jan 23

Trygve:
What integration is actually needed? Why not just copy them over, link to them from the front page and be done with it?
Jan 23

Trygve:
I just tried to access this: http://www-mockup.opencsw.org/packages, but that doesn't seem to work. Isn't getting that page to work + /maintainters just about all that's needed to go live?
Jan 23

You:
william: I have to remove headers/footers and I have to redo the bits that access the db and renders the html.
Jan 23

Trygve:
Why? It work as it is today, why change it?
Jan 23

You:
maciej: Sounds like you have to redo everything.
william: It's not a full rewrite, it's moving from one PHP environment to another.
trygve: We need to change it because it looks differently, you'd have the old look and feel of the website.
Jan 23

Trygve:
The CSS for the site looks semi-decent. I can probably spend some time on that tomorrow unless there are other issues that you want to prioritize higher.
Jan 23

Trygve:
Stuff looks horrible as it is, I'd say moving it is pri 1, getting it to look pretty is pri #2 or #3.

The point is that if we leave another meeting without getting it done it will just slip until the next meeting and so on.
Jan 23

You:
william: We can move to prio 2, but we have to be aware that we're going to see the old site for packages.
maciej: I'm fine with seeing the old site for packages for a couple of weeks.
dam: +1, when it gets annoying, it gets done
maciej: Can you let Apache handle /packages.* just the way it's handled today?
Jan 23

Trygve:
Handled by apache, is there something special there that's needed?

I know there is a redirect from /packages.php => /packages
william: Yes, I think so.
Jan 23

Trygve:
dam: exactly! We need to create incentive to keep on working with this. Doing the switch will be a huge incentive.
Jan 23

You:
dam: :-) (drinks something horribly sour and makes a face)
william: it's almost done
maciej: "almost" is deceptive
william: ok, it can't connect to the db
maciej: why?
william: db from Phil has not been copied, or maybe the db name is different in the dev zone
Jan 23

Trygve:
Can't we just go directly to the production DB? We only need a read only user.

skayser: aaaah, funkzioniert (about sudo on web dev zone)
all: yey!
william: I only need to check the database name. Few minutes.
skayser: passwords are there
maciej: what's the hostname?
skayser: ocswchbiesv04.opencsw.org
william: I'm about to install some packages.  How should I report it?
maciej: send to the buildfarm mailing list?
maciej: (needs a break)

skayser: Which languages?
william: PHP. So far we stick to PHP. We could also use Django but only for middleware.
william: Should we proof-read and check the wording and organization? All the items we have are technical.
maciej: Let's not touch the content yet. Content should be lev 2 or 3.
skayser: We 've already touched some content.
maciej: But not much. Only where we have too.
skayser: We don't have sudo access, because it asks for passwords and we don't have passwords, there, the auth is via ssh keys.

Jan 23

Skayser.de@googlewave.com:
arghh this new website discussion is a bitch, we need someone to take charge of it completely ...no endless coordination and fiddling with various items assigned to various people

Jan 23

You:
Hard to achieve.  William is the main person...
Jan 23

Skayser.de@googlewave.com:
There is also the challenge that there is three dimensions to it: technical, content-wise/psychological (WRT to phil), design
Jan 23

You:
maciej: It's best to separate them. Like in a battle: You provide cover fire, I run. Then we switch.
william: I think the wording of all the content is very important. The first impression that people get might be the last. It might be not a lot of work to do, but it might be very good to do 2 steps: a short prio 1 review, and then longer review prio 2. The design is pretty clean. If you go to the extend-it section...
Jan 23

Trygve:
The content is important, but it is not that important just to do the switch. Once we have a platform where it actually is possible to change the content, people will contribute. Anything is an improvement over what we have today anyway.
Jan 23

You:
There seems to be agreement that it's the right thing to do.
dam: (shows something to william on his lappy)
maciej: We can do a switch without announcements, and once the content is ready, we announce the new website.
william: We can do it even after the announcement. We just need to make sure there's nothing stupid in the first version. But most of the job will be done after the announcement. But I would like there to be a short review. For instance the history page. If you look at the new website, extend-it, hover over contribute. This is messy.
all: (agree)
william: It's better than what we had before, but it has to look finished.
skayser: The real challenge to me is working on something, and fighting with Phil at the same time. I would love if Dago would provide cover while we work on the website. I could easily work on that, but I don't want to discuss every sentence with Phil.
william: I'll do all the PHP work, but I cannot work on the technical part and the content at the same time. As soon PHP will be integrated, I can do other things.
skayser: We could ask Ben if he could take care of the "contribute" section. Then it becomes clear, you (william) do the technical, maciej checks the URLs, I look at content
william: I understand why I had unexpected errors, we have been using mysql from Sun...
skayser: I'm on vacations next 2 weeks, I worked with Ben, he was the last to put the words on in the right order.  It's best to talk with him.
Jan 30

Ben W:
Is this still required?  I'm able to put some effort into website stuff this weekend.  If someone sends me links of things to proof/correct, that'd be easiest, but I can simply browse through mockup and correct as I notice things.
Jan 23

Skayser.de@googlewave.com:
any CI/web designer listening?
Jan 23

Trygve:
CI?
Jan 23

Dagobert:
CI = Corporate Identity, make uniform design with logo for website including coloring etc.
Jan 23

You:
(17:56 on Saturday)

dam: Let's looked at the minutes from the last minute and see where we are. How much work has to be done?
dam: We might ask maintainers to add license tags.
skayser: svn blame?
dam: Yes!
skayser: Crawl all the packages, build a list for each person that they were involved in, and ask them if they are fine with adding the license
maciej: Will it be the CDDL license or GPL?
william: CDDL was friendly for build descriptions because it's more permissive and it's Sun friendly, close to Apache license. Maybe it will help to have some of the work compliant with what Sun has.
Jan 23

Trygve:
I would like CDDL over GPL (I have issues with the GPL).

What is SFE using?
Jan 30

Ben W:
And I'm a GPL kinda guy... :)
Jan 23

Dagobert:
(Looks at /export/mirror/sfe/) On the SF page it says GPL: http://sourceforge.net/projects/pkgbuild/develop

maciej: I used GPL on mine, it's the default license I use.
maciej: Is there a concensus?
skayser: Not really.
william: There's work to be done to assess the implications. We'll be able to do the decision afterwards.
maciej: There is software with dual licensing: GPL and Apache, etc.
skayser: There's no strong opinion.
william: I don't know if we got enough information, and we didn't work on this subject since Oslo. I'm not sure if we can reach a decision now.
all: (agree)
maciej: So what do we do with this topic? Put it on a shelf for somebody to pick up?
Jan 23

Trygve:
Sounds good for me :) We have more important stuff to worry about for now.
skayser: yes

dam: pkgrequests list is there, but not much traffic
Jan 23

Trygve:
That's so that external people can request packages, right?
Jan 23

Dagobert:
Right. Usually these get audited and added to http://www.opencsw.org/pkgreq 

maciej: Do we need to do anything?

skayser: It would be good to have all maintainers on the
Jan 23

Trygve:
Hot topic!

The standard quote is "TOO MUCH TRAFFIC" from phil and grey beards.
 pkgrequests and bug list.
maciej: I'm likely to filter the bug list out.
skayser: That's fine. But we can postpone it until after we redo the website.

dam: Clarify the maintainer status.  I write to maintainers who are silent, and usually they get the retired status. But there are maintainers who are marked active, but aren't really active. I would like a more conservative approach. If they aren't really actively contributing, their status should be changed to idle.
maciej: Any actions that you suggest?
dam: There's a concern that it would look bad to have packages assigned to retired maintainers.
skayser: Can we have a QA group? There might be no one looking at some packages, but if there was an important bug report, somebody would pick it up.
maciej: We talked about tiering.
dam: Even for tier 1 packages, we don't have maintainers. It's not enough to say that a package is tier 1. Somebody has to do the work.
dam: william, do you still do the statistics? Can I have a look? Are they current?
william: Yes, http://www-mockup.opencsw.org/get-it/package-statistics/
william: The assumption is that the package versions between sparc and i386 are the same.
dam: Some packages might be released only for Solaris 10, and not necessarily for Solaris 8. See DTrace toolkit.
william: There are just a couple of fields... (explains...)
dam: The numbers are high... wow.
dam: Let's go back to agenda.  Despite the fact that we have a high update rate, and we're ahead of Blastwave in terms of updates, we're still falling behind in about 60-70 packages, mostly php.
william: Can I go back to statistics? Currently, stats are only about package releases. There are more statistics in the db, which aren't displayed yet.  I've got stats about packages, I'm populating counters in a table dedicated to single package statistics.  For each package, I'm computing its last update time and the number of times it's been updated. Each time the package gets an updated, the counters are increased. For instance... (reads some stats) 
dam: You should make nice pictures from it on the package pages.
william: What would you like to display?
dam: (shows activity graph) in a very smal catchy picture
maciej: perhaps Google Chart API can be used?
william: 2 concerns: first is that I have to call a remote service and it's included in the webpage, in most cases they are opened maybe once a day. I would generate png and store them until they go out of date. And the URLs would be very long, because they contain data.
maciej: What about new small graphs? You have to write code
william: I already have a library to do that, and it's little code to write.
amaier: And it can be cached by php optimizer.
william: I can show you the code which generate the pngs. I only call a method which renders the graph.  We have to define what to display about individual packages.

(the dinner topic kicks in)

dam: Dinner at 20:00? Short break now?
all: yes
Jan 23

You:
18:44

dam: What does lutefisk do?
william: It's a tool that reports stats about package installations and mirror usage. It's like popcon in Debian.
dam: So people have to install it locally, and then we get more accurate statistics. How does it work?
william: It's using a package hook. From the client side it's a simple script, which is called by pkgutil when it has finished fetching data from the mirror.  There are two flavor: med smor and med tilbehor. There are 2 different kinds of statistics. One package for each kind of statistics.  Perhaps the user would like to report one statistic, but not the other. On client side, there are two scripts, one for each package. Those scripts make calls to the opencsw server, and information is in the URL, for instance that the package version is this and that. We should be able to know if we still have so many people running Solaris 8 or 9. If we only keep track of the version of the package we deliver, most of the packages are provided for Solaris 8. So we have to keep track of both OS version to which it was downloaded, and OS version it was built for. We'll have a quick look at the DB shortly. The other lutefisk flavor reports the mirror information: base url, architecture, and host OS version. On the server side, it relies on MySQL and PHP scripts, MySQL provides 3 tables for the content, and PHP scripts process the arguments from the called URL and identify counters to increment. There are a few known issues, it's only 0.1 versions. There are 2 possible problems: a DoS attack and spamming.  There are a few controls on the server side, for instance we won't compute statistics on packages that don't exist. There needs to be flooding detection.  It's not a problem though yet.
dam: Some client authentication is important.  For each user, there should be some sort of account.
william: If people accept to take part in package survey, it has to be very simple. As simple as pkgutil -i ...
benny: Users won't like to be indentified with a hash, or anything.
william: If you put the hash into the URL, it'll be less anonymous. Hashes would be in the Apache log.
dam: It would help to see how popcom does it.
william: (shows db tables)
all: mmm... statistics!
Jan 23

You:
19:08

skayser: What now?
all: we're tired
dam: Dive into the other things?
skayser: Testing? Part of the release branches and semantics.
dam: We had the discussion about when there's a new package and it's rejected because it must be tested more. testing/ contains too much stuff, and packages don't get tested because you can't keep installing from there. We could have: /home/experimental/<project>/... and you would subscribed to mirror.opencsw.org/opencsw/experimental/<project> --- then they can go into unstable/ which means "it's a finished package that could be released".
skayser: Why not testing/?
Jan 23

Trygve:
How about everyone that feels like it just create a directory under testing/?

I'm sure Dago's tool to create the catalog can be tweaked to create a catalog per directory. Ala:

find /export/testing -type d
dam: well... we could, but we already have testing, that would be a redefinition.
Jan 23

Trygve:
Not really, just an addition. testing/ as a concept still stands, but it's just a bit more segmented.

Small stuff, updates etc should go directly into testing/

The main point is that with this small change we can get going without introducing big changes that involves lots of discussion. It's nice getting some experience with the different ways to solve/improve the situation.

maciej: (presents a proposal)
all: we're tired
all: (go to eat dinner)
Jan 27

You:
Hacking session.

- end-to-end tests, the first version (by maciej)
- R (by dam)
- fontconfig cswmigrateconf (by amaier)
- GAR color output (by skayser)
- ... (by william)
- ... (by benny)
Jan 27


OpenCSW Wintercamp in Munich, Day 2

Attendees:
dam
wbonnet
maciej
gbensons
skayser
amaier
Jan 24

You:
The package database by wbonnet

wbonnet: (presents a slide with db schema) I started to work on perl script to populate the database. The code is in SF. The model is not finished yet, it's a work in progress, I can populate about half of the model. In this part (slide) of the model, I've got a table to get the maintainer in the databse, it's meant to replace the current db. It's intended to use the ldap directory, and this db will have only the minimal info, everything else about the maintainer will be in ldap. The idea is that there will be the same kind of information. I've got several entities, the package entity is the general entity to model the package. This entity is not a file.
maciej: Can we get rid of the "package" word?
wbonnet: Not easy, it's about packages. I have to use other technical terms also. I don't see the need to remove this word. It's a package database.
Jan 24

Trygve:
Random comment:

Now that we're starting to talk about releases, I think it's important that a package always is package name+release. If not it will be hard to migrate over time.
Jan 24

Dagobert:
Yes. This has already been adresses by reporting bugs always against a specific revision. Also it is then possible to show all open bugs in a specific release.
maciej: You have to trust me. It's confusing.
wbonnet: What do you want then?
maciej: Um... it's a hard one.
wbonnet: Maybe "catalog entry"?
maciej: (shows thumbs up)
wbonnet: There will be an association of the current maintainer or a team of maintainers for a given catalog entry. There will be a LUT for maintainer status. There are entities for upstream information and gar information. There will be information about the current upstream version and the gar version, and versions in the stages. But I need to find a way to model this. Upstream watch will use this information. What is currently defined in Makefiles right now as upstream info, might be later in the database. It would be useful to have this because it would make the processing much more faster. Working with svn+gmake is pretty slow. If we had this in the db, we would also track upstram info for non-garified packages. Maybe we can define some defaults, for the non-garified packages. All the scripts to create the model are available. For each catalog entry I keep track of the version information of this entry. Then I have information about the files. There's information from the package, close to pkginfo. There is information about the filename, install size, version, revision, checksum (md5/sha1), description, maintainer info (the one from pkginfo), a link to the bugtracking URL. From there there is an association to architecture.
maciej: Different architectures have different files.
wbonnet: I know, will fix. There will be associacions to 3 entities: File, directory, symlink.
maciej: Can prototype and the file content different?
dam: They can't. You can't run pkgmk on source with mismatches.
wbonnet: File/dir entities will have all the information. This allows much more information in the package page. It will allow people to check for package integrity. Also useful for checked for files modified from version to version.
maciej: What is the primary key of the file entity?
wbonnet: Just a counter. There can be different instances of the same file with different permissions, even though the content is the same. Once we have all this information in the db, we can detect conflicts.
maciej: Can we have binary-sharedlib association stored there?
wbonnet: I will add it.
dam: How about RUNPATH?
wbonnet. Will do.
dam: Also sonames for shared libraries.
wbonnet: OK.
wbonnet: Dependency is a common association. I would also like conflicting packages information. I would also like to have information about package providing a service. Useful for database servers. You might install a PHP application, and you need any SQL server.
maciej: Any usecase for it?
wbonnet: Potential support in pkgutil, later. Initially the association will be empty. Next, I will need information from other databases, for instance mantic, for interfacing. Also with the stats database.
maciej: How do you want to synchronize/crossreference them?
wbonnet: Two questions.  I will need to have an ID in the package, but there's also a question of should we have distinct databases. Is it going to be heterogenous (mysql/ldap) or not. Just one db is easier to do.  The code is in SF in http://spmt.sf.net - although it's not finalized.
wbonnet: The next part of the project is XML catalog. It has models of sources and a model of packages. I work on a tool to create those catalogs. It's an alpha version.
gbensons: What's the benefit of XML?
wbonnet: Several benefits: it's possible for any tool to read the format, and it has much more information.
gbensons: Would you have two catalogs then?
wbonnet: Peter was working in an XML format, we would merge this. We talked about this in Oslo. I have to check with him on the progress. We would like to have a single package project. Having this would be interesting, because we could implement more functionality in pkgutil, and having more detailed catalog would make it possible to have a command line tool equivalent to pkg-cache, or other cache tool. If there's always information in the catalog, with a signature, you would be able to query which packages contain which files, do consistency checks.
dam: basically, pkgchk from another source
wbonnet: Yes. Another benefit is that it supports several sources from the beginning. You could have extra sources with other packages, say, SUNW. There would be no maintainers for source packages.
maciej: pkgutil supports multiple sources already. What's the difference?
wbonnet: in XML you have a schema which defines all the sources in the same source catalog, and for each source you have a... (distraction)
dam: If you use one source, other source gets thrown away
maciej: AFAIK, it's not the case, pkgutil merges the catalogs and uses the newest version
dam: Now you have to know URLs, the XML description knows the URLs and refers to abstract names. It could also have mirrors?
wbonnet: Sure. I'm not currently working on a tool to use this, the priority is the database.
maciej: I'll write code to populate the shared lib dependencies.
dam: There's also james' schema which he hasn't shared yet.
maciej: The one which catches traction first, wins.
wbonnet: If we have access to the other schema, we can compare and merge.
all: yes

(lunch)
Jan 30


You and Dagobert:
Staging by maciej
maciej: stable is dead. We need a way of updating it. The previous method of individual testing is too tedious and that way we will never again have a new stable. -> We need a new method of making stable

maciej: new stable may be snapshot with no reported errors. broken packages should be removed from the catalog if noone takes it over. The idea is to have a frozen snapshot that has been running for 3 month and that has no serious bugs. What do we have to do to achieve this?
Take packages from testing/ and move to unstable/
Jan 24

Gbensons@googlewave.com:
correction. testing/ moved to experimental/. unstable/ will be empty for the initial migration

TBD: How does the new process work

file
 

Working towards the new process
Create experimental/ with subdirectories
Copy everything from current/ to unstable/
Potential announcement for people who subscribe to unstable
Move from current to time tagged frozen directory (e.g. snapshots/2010.03)
Jan 24

Trygve:
I would like it if the naming was YYYY.MM, just because that's what OpenSolaris use.
Jan 24

You:
Sounds good, edited.

Symlink from testing to snapshots/2010.03
current/ becomes a symlink to testing/
Milestone: New layout achieved
The release process
New individual packages get released into unstable/ just the way they've been released into current/ in the past.
Packages in testing/ only get bugfixes, without version upgrades. Security issues and significant bugs are fixed. This continues until the testing/ branch is free of significant bugs.
Once the testing/ branch is free of significant bugs, the procedure of upgrade from the old stable to the stable candidate gets tested. Identified issues get either solved, or documented. An upgrade document gets written.
Tier 1 packages get tested by respective maintainers: does the configuration migration go OK, and does the application actually work after the upgrade.
When the upgrade procedure is tested, and the right time comes, testing/ is promoted to stable/ and unstable/ gets snapshotted to testing/
GOTO 10


Things to be solved first:
Write up the plan in detail (maciej)
Make tool to generate catalogs out of nested directories (-> Dago)
Synchronize versions of all released packages to be visible in Mantis (-> Sebastian)
Bugs can be reported against specific package versions
It will be possible to make a list of all open bugs in a catalog like stable/
Implement tier (1/2/3) information storage in mantis (skayser)
Assign tiers to packages (coordinated by dam)
Reorganize the mirror structure (-> Maciej)

maciej: package pages (web pages) should have tier information
maciej: the priority field can go away from mantis
dam: mantis should contain the many-to-many relationships of package revisions and bugs. This would allow to calculate which bugs relate to which branches (unstable/testing/stable)
skayser: What's the mapping of severities?
all: (feature, trivial, text, tweak) -> feature; minor -> minor, major -> major, (crash, block) -> critical
skayser: We need a field for the release manager to identify bugs that must be fixed before stage promotion. Which one do we use?
maciej: Tags. We can have for instance "required-2010.01" assigned to a package.
maciej: We need a good writeup of this proposal.  I can update the wikipage.  http://wiki.opencsw.org/automated-release-process.
maciej: Nice, let's move on to the next topic.
Jan 24

You:
OpenSolaris experience with OpenCSW packages, SVR4 to IPS converter 

dam: trygvis has done some work on it. The idea is to have a native backend in GAR for IPS. It needs to be done.
wbonnet: I have OpenSolaris on a netbook, I can test it.
dam: We have OpenSolaris on the buildfarm, we can start generating this backend. We should create a new repository with IPS packages.
maciej: What about packages that aren't in GAR.
dam: We have the converter from trygvis. The IPS repository will look differently anyway. Blastwave packages are only conversion. We should look for the best possible integration into OpenCSW. It might even a different set of packages. If it's only a copy, there's no advantage in using IPS.
skayser: Was there stuff that didn't work in OpenSolaris?
dam: No postinstall scripts
Jan 24

Trygve:
My suggestion here is to enhance pkgutil to execute the scripts.

When we're creating the IPS packages we'll stick the scripts somewhere and pkgutil can just execute them. The preinstall might be hard to do, postremove too perhaps.
Jan 24

You:
dam: Do we really need preinstall? Can't we only use classutils?
Jan 24

Trygve:
Dunno, perhaps not
skayser: What happens to the postinstall?
dam: For svr4, it gets executed, for IPS it doesn't. Now there's svr4 + IPS, in the future it will be only IPS.
wbonnet: We can have some packages built on OpenSolaris. I could compile Firefox 3 for example.
dam: Build machines will be available pretty soon.
dam: Should we promise to deliver the same content for OpenSolaris as we're providing for Solaris 8/9/10, or should we start with a completely different repository?
wbonnet: Completely different repo will break the software stack consistency. We can't go with something completely new on OpenSolaris.
dam: OK, you're right. Conversion is just as good.

Jan 24

Dagobert:
William has offered to host the Summer Camp 2010 in Paris.
Jan 24

You:
Meeting on Thursday evening would be also good. We would have one full day more: 2.5 instead of 1.5.

Jan 24

You:
Source packages

dam: There's a target in GAR already, set nosourcepackage = 0 and the package will get created.
maciej: What's the use case on a blank Solaris 10 box?
dam: Install the source package, cd /opt/csw/src, you need root.
skayser: For Debian, you can say dpkg source and it installs in the current dir.
dam: Not done yet.
maciej: What's your goal?
dam: Compile real Java on the site.
skayser: Build software with other flags than the current ones. Flexibility.
dam: Rebuild part of world.
skayser: Like the guy with the nagios packages.
maciej: Oracle perhaps?
dam: Absolutely. When we don't have the distribution rights, we offer the build receipts. Acrobat.
maciej: corefonts?
dam: We don't have rights?
maciej: Debian install pulls from Sourceforge.
Jan 24

You:
Featured articles

skayser: wbonnet said that we could have featured articles, describing usecases.
dam: "Deploying monitoring with OpenCSW"
skayser: Building Oracle from source packages. When we introduce those features, we could publish articles.
wbonnet: Articles about problems that users can have, setting up nagios, or tomcat cluster, they might become interested in our software.
maciej: In the past, I published blog posts.
dam: Something like that. Great.
wbonnet: This is one of the purposes of Wordpress.
maciej: Also notes with links to articles elsewhere.
Jan 24

Dagobert:
Stuff to be on the package info page
Chart with open bugs vs. time
Releases marked as red bars
Graph of dependencies made with Graphviz
Download the package directly, additionally pristine sources and patches
Download package with all dependencies as datastream
Add link to the svn repository containing the build recipe
Jan 24

You:
It would be fun to have it labeled with an innocent "How was this built?"

Open bugs
Which catalog contains which release