code repository: http://code.mios.com

Developer's Corner General Dev Discussions
1

Hi everybody.

As some of you have seen, the code repository at http://code.mios.com is running. It’s still in somewhat ‘evaluation’ mode; but so far is technically satisfactory.

Please note that this is a developer’s tool; a novice non-technical person might find it confusing. Our vision for the plugin creation/collaboration/publishing/distribution includes at least the following pieces:

[ul][li]For the end user: The “MIOS Marketplace” window in each Vera. Here’s where anybody can pick and install plugins, devices, and other optional features.[/li]

[li]For the publisher: An easy to use frontend to put some code in the marketplace. Still missing. Currently, we only put anything in the marketplace after QA tests it. We’re discussing about a ‘beta channel’ that would be more permissive.[/li]

[li]For developers: the code.mios.com repository. This what I’m talking about here.[/li][/ul]

This is just a service we provide to our users that are willing to develop on MIOS software (ie: Vera), simply to have a single place to host everything. Of course, anybody is free to use Sourceforge, Google Code, GitHub, etc.

I’ll soon write some wiki pages as a guide, but first want to share a draft with you, hoping to get some comments and other feedback:

[ul][li]Everybody is welcomed here. Just ask for an account or a project to be created.[/li]

[li]There are many projects hosted there, most initiated at user’s request.[/li]

[li]Each project has a source code management server (subversion currently, soon maybe Git and Mercurial) and a Trac instance that supports the project management with a wiki, code browser, bug ticket system, etc.[/li]

[li]All free projects are world-readable. Commercial projects are free to choose if they want to be readable or not.[/li]

[li]There are four roles: Anonymous, Collaborator, Project Owner, Administrator:
[list]
[li]Anonymous is anybody, no need to register. Can read the project’s wiki pages, browse the code, checkout the code using Subversion, and create tickets.[/li]

[li]A collaborator is a registered user that collaborates in a project. Can handle tickets, contribute to the wiki and commit code to the SCM.[/li]

[li]The project owner is (usually) the user that asked the project to be created. Has Trac admin rights to add other users as collaborators, can also delete wiki pages, and other special tasks.[/li]

[li]The administrators are MIOS staff, we create the user accounts and projects.[/li]

[/list][/li]

[li]If you want to collaborate on an existing project, ask the project owner.[/li]

[li]We’ll try to avoid duplication between the forum, the wiki and the code projects. At the same time, we have to fight fragmentation and keep everybody interested well informed. Some ideas for this:
[list]
[li]There will be a single wiki page listing all projects with a short paragraph explaining it, and specifying the project owner and collaborators.[/li]

[li]The project should be fully documented in the project’s wiki pages. I’ll try to put links to those pages on relevant places.[/li]

[li]Use the project’s ticket system to communicate bugs, missing features and wishlists in general. Remember that anybody can create a ticket without registering.[/li]

[li]Use the forum for announcements and to ask for advice/support.[/li]
[/list]
[/li]
[li]There’s not yet any link between this code repository and the MIOS Marketplace. When a project reaches a ‘releasable’ status, the project owner should ask an administrator to check out a specific revision and pass it through QA and eventually to the marketplace.[/li]

[li]We can’t give any guarantees on the service availability or stability. I’ll do my best; but it’s conceivable that a project could have to be deleted and rebuilt from backup and sometimes the commit history is lost on this process.[/li][/ul]

This should cover most of the important points. Any comments, suggestions, requests, etc?

2

Javier - This is a good start, one thing you may want to do when collaborator submits a project and its posted to the repository is create a sub forum for the plugin.

i.e. Plugin Discussion Board
Sub boards
Paradox Plugin Discussion
Smart Phone Discussion
Somfy Plugin Discussion

This will allow you to keep the boards cleaner andmore organized.

3

@javier, can you change the SVN endpoint URLs that are listed in URLs like:

http://code.mios.com/projects/8/help/svn_access

listed in Repository Hosting

these need to be changed to reflect the “[tt]code.mios.com[/tt]” URL style instead of the “[tt]mios.repositoryhosting.com[/tt]” style indicated there, since that service doesn’t exist.

Also, is there a particular Style of sub-directory structure you’d prefer us to put things into as a standard template/recommendation?

For the Weather upload experiment, I just put them all at the root, but I imagine a more consistent representation/structure would benefit all.

4

[quote=“guessed, post:3, topic:165656”]@javier, can you change the SVN endpoint URLs that are listed in URLs like:

http://code.mios.com/projects/8/help/svn_access

listed in Repository Hosting

these need to be changed to reflect the “[tt]code.mios.com[/tt]” URL style instead of the “[tt]mios.repositoryhosting.com[/tt]” style indicated there, since that service doesn’t exist.[/quote]

I don’t get you. which url’s are you seeing? where? how do you think should be?

[quote=“guessed, post:3, topic:165656”]Also, is there a particular Style of sub-directory structure you’d prefer us to put things into as a standard template/recommendation?

For the Weather upload experiment, I just put them all at the root, but I imagine a more consistent representation/structure would benefit all.[/quote]

So far, most projects fall on one of three cases:

[ul][li]Device Plugins: Usually these are exactly 3 XML files (D_xxx.xml, S_xxx.xml, I_xxx.xml). I don’t see a need for directories.[/li]
[li]Pure Plugin: Usually a single Lua file.[/li]
[li]Lua Modules or code snippets: Here some subdirectories can help to join related modules, but most should stand alone.[/li][/ul]

In any case, subdirectories would help when you go beyond the basic (minimal) amount of files. For example, in your weather plugin: if down the road you want to put some functions in a Lua module, it might be better to separate on one directory the Lua module(s) and on another the plugin itself.

5

When I run this URL:
http://code.mios.com/projects/8/help/svn_accesst

from this page:
Repository Hosting

it shows the SVN link as:

svn checkout https://[b]mios.repositoryhosting.com[/b]/svn/mios_panasonic-tv panasonic-tv

where the SVN repository is listed as [tt]mios.repositoryhosting.com[/tt] instead of [tt]code.mios.com[/tt]. Anyone “copying” this URL will get an error when they attempt to attach an SVN Client (as I did)

On sub-directories…
My logic here is fairly simple. The more “structure” we supply now, the easier it will be to automate the Plugin packaging down the road.

Most projects usually end up with readme.txt, images etc that may or may not be packaged into a target Plugin deployment descriptor (whatever form that eventually takes) … without the complexity of having to build a makefile (or modern equiv)

6

I agree putting some sort of wrapper to signify what is the plug-in and what isn’t would be nice. In my Wake On Lan plug-in I put the three files in a folder and am planning on putting readme and other like files as a peer of that folder. It might be nice to name the containing folder to signify it as the container of the real plug-in files to help with an automated deployment like Guessed suggests.

How about naming the folder with a .plugin extension? This would be similar to how bundles would in Mac OS X.

7

[quote=“guessed, post:5, topic:165656”]When I run this URL:
http://code.mios.com/projects/8/help/svn_accesst

from this page:
Repository Hosting

it shows the SVN link as:

svn checkout https://[b]mios.repositoryhosting.com[/b]/svn/mios_panasonic-tv panasonic-tv

where the SVN repository is listed as [tt]mios.repositoryhosting.com[/tt] instead of [tt]code.mios.com[/tt]. Anyone “copying” this URL will get an error when they attempt to attach an SVN Client (as I did)[/quote]

oh, yeah, the green question mark. you’re right, i’ll try to get if fixed.

BTW, i didn’t notice because i’ve just used the URLs that get displayed under ‘Access URLs’. simply copy those to SVN

[quote=“guessed, post:5, topic:165656”]On sub-directories…
My logic here is fairly simple. The more “structure” we supply now, the easier it will be to automate the Plugin packaging down the road.[/quote]

right, but making non-structured projects to conform with some extra structure that we later define might be easier than transforming different styles to any ‘official’ one. For now, i think just tying the 3 xmls on a directory (like mike did with wake-on-lan) is enough.

besides, in any decent packaging system i’ve used, you end up having to specify each included file. just like a makefile or a manifest.

for documentations, the project’s wiki would be much better than a readme, don’t you think? especially since there shouldn’t be installed manually, so the docs would be about usage, not installation.

about adding images (or other extra ‘binary’ files) to the project, there’s a shared ‘drive’ facility (a WebDAV storage) that i can activate on those projects that need it and want to keep those out of the SCM repository.

8
BTW, i didn't notice because i've just used the URLs that get displayed under 'Access URLs'. simply copy those to SVN

That works until I try to checkin, since they’re the “public” view…

besides, in any decent packaging system i've used, you end up having to specify each included file. just like a makefile or a manifest.

Going back to my original goals, I’d like to make this as “prescriptive” as possible so that folks “just starting” don’t get put off, and have a place to start.

I imagine that a standard directory name, pre-created during the SVN Project provisioning process, will save people a few headaches later on (where a packaging could be more automated for them without having to build manifests and the like… you’re already walking deep into “core” developer territory, which I believe we need to steer clear of to enable wide-participation.

for documentations, the project's wiki would be much better than a readme, don't you think?

Yup, but you need to prescribe something …

For the “released” part of the project, we’re already using the Wiki.micasaverde.com site. But often there’s stuff that needs to be project-attached.

The better the “template” (prescriptive) this is, the more likely people will conform (not a requirement) but also that’ll lower the barrier to entry for folks writing simpler stuff.

Again, key to building out the base of people writing things.

9

Unfortunately, I’m busy this week, so don’t have much time to start working on the plugin(s) right away. Although, I played a little bit with the new code repository over the weekend, I hopefully will have more time by the end of the week…

BTW, is there any way to quickly subscribe to a thread w/o posting?

10

If you click the “Notify” button on a thread you will be subscribed to that thread.

11

I must have been blind! Thanks. :slight_smile:

12

[quote=“guessed, post:8, topic:165656”]

BTW, i didn’t notice because i’ve just used the URLs that get displayed under ‘Access URLs’. simply copy those to SVN

That works until I try to checkin, since they’re the “public” view…[/quote]

if you’re logged in, it displays a ‘Public’ and an ‘HTTP’ URL (not the best name), use the second one for developing.

About all the ‘prescriptive’ comments, i concur with some of them; but remember that a Trac/SVN repository is a tool for developers. any ‘end user’ code submission tool would have to be a lot simpler, more like a single form where you fill ‘name’, ‘description’, ‘type’ fields and the files themselves.

13

Is code submission a developer task? If you have code to submit, I would say you are a developer.

14

sure; but SVN and Trac are definitely not as beginner-friendly as we’d like to. that’s why i talked about three different pieces: MIOS Marketplace to download, an ‘appstore’ frontend, and developer tools.

15

Thanks Javier, this is a great start for plug-ins and should help a lot of people out.

It would be great if we could add some of the lua code snippets found in the forums, and if you could use that opportunity to make any code corrections or optimizations. Mostly just for examples, but they could cut and paste directly too.

Thanks for this MCV and Javier!

16

Hi all,

There has been some code activity the last few days. Great!

Now, one of the things we want to avoid is information fragmentation. There are several communication tools, and we want to make it easy and natural to find anything. So, this is what i propose:

[ul][li]The project owner should create a ‘topic’ here in the forum (one for each project). To make those easy to recognize, the title should be prefixed by [tt][CODE][/tt]. There would go any ‘informal’ conversation about that specific project. The first post should include a link to the project.[/li]

[li]There’s a wiki for each project, it should be used primarily to document the project: what does it do, how to use it, etc. The first page should include a link to the respective forum topic.[/li]

[li]There’s also a ticket system for each project. It’s meant for both bug reports and patch proposals. If you attach a diff file with your suggestions to a ticket, it’s quite easy for the owner/collaborators to review it online.
Unfortunately, it seems that ticket submissions can’t trigger email notifications; but there’s a RSS feed on the ticket reports. The project owner (or anyone interested) can add to any RSS agregator to stay informed. Still, a ticket submitter could also post on the forum topic to be sure that it’s read. (I’ve already submitted a ticket on a project, can you find it?)
But please, don’t rely only on the forum. The big advantage of the ticket system is that it’s persistent and stays ‘open’ until it’s fixed (or rejected). A forum post, OTOH, is hard to relate to which version is referring, or if it has already been taken care of.[/li]

[li]There’s some integration between the tickets and commit messages ([url=http://code.mios.com/help/post_commit_messages]http://code.mios.com/help/post_commit_messages[/url]). this makes it’s easier to keep tickets relevant to the latest code version.[/li][/ul]

Any suggestions?

I’ll start creating topics for the generic Lua utilities and SmartPhone plugin.

17

Hey Javier,
Is it better to prefix everything with [CODE] or to create a specific named Forum area for Plugin and/or Library announcements?

I’m assuming the prefix is being used to help “find” the Plugin announcements within the jungle of discussions/postings under Luup (using Search, or Sorting, of that Forum category)

If you created a separate “Plugin Announcements” section, which is User populated/created, you could achieve the same. Anything posted OT would be moved to the correct category.

This will make it easier to find stuff for people not “living” in the Luup Development category, and would act as another advertisement for the functionality (with the initial posting containing hyperlinks to the Project/Library/Plugin Wiki main page as well)

It also avoids the term CODE (implementation) since you’ve already spent time calling these [largely] Plugins (functional) to make it simpler for users (as opposed to Developers).

The other comments are fine.

A few extra questions:

a) What vehicle should be used to “publish” release-versions?
Obviously we don’t want folks just pulling the latest version from Source Control, since those files may be a WIP. A few people are using Box.net accounts (etc) for published versions. I think it would be useful to formalize the “download unit” (or Derived Object/DO and mechanism) until the Marketplace is formalized.
ie. a named ZIP, what convention, where etc?

b) Is the Wiki formatting engine consistent between wiki.micasaverde.com and code.mios.com?
I have a bunch to transcribe, so it’ll help to know the pitfalls in advance.

c) Failing (b), do you have a template for homepages?
These have a tendancy to get all over the map (like all Wikis) without some structure being put in place. You can see the type of thing I’m referring to in my examples on wiki.micasaverde.com (for Paradox Security, Weather etc)
A template helps consumers “see” a consistent version of the world, even if there are different people producing the Plugin, and should ease the uptake.

18

[quote=“guessed, post:17, topic:165656”]Hey Javier,
Is it better to prefix everything with [CODE] or to create a specific named Forum area for Plugin and/or Library announcements?[/quote]

this is a personal preferences issue, at least until we get some good or bad experiences. personally, i’m reluctant to crowd the top level. in any case, it’s not so hard to change/reorganize after a few topics get created.

right.

[quote=“guessed, post:17, topic:165656”]If you created a separate “Plugin Announcements” section, which is User populated/created, you could achieve the same. Anything posted OT would be moved to the correct category.

This will make it easier to find stuff for people not “living” in the Luup Development category, and would act as another advertisement for the functionality (with the initial posting containing hyperlinks to the Project/Library/Plugin Wiki main page as well)

It also avoids the term CODE (implementation) since you’ve already spent time calling these [largely] Plugins (functional) to make it simpler for users (as opposed to Developers).[/quote]

right now, i’m more concerned about communication between developers, to encourage collaboration. it seems that Trac is conceived to be used together with either a mail list or a forum. it’s very common on other projects to talk mostly on the maillist, but constantly refer to tickets held in Trac. I think it’s a good model.

to notify non-developer users, yes, maybe some “Plugin Announcements” would be best. this section would be very sparse at the moment, but i’d love to see it gathering steam… of course, code talk should be (mostly) kept out of there

[quote=“guessed, post:17, topic:165656”]a) What vehicle should be used to “publish” release-versions?
Obviously we don’t want folks just pulling the latest version from Source Control, since those files may be a WIP. A few people are using Box.net accounts (etc) for published versions. I think it would be useful to formalize the “download unit” (or Derived Object/DO and mechanism) until the Marketplace is formalized.
ie. a named ZIP, what convention, where etc?[/quote]

i’m in the ‘design/daydreaming’ phase of a download tool, written in Lua, used mostly for beta-testers of plugins. the developer would write a simple ‘manifest’, and add it to the repository. the tool would read it, download any changed file and record it. next time, it could check any project that it has installed, and update all.

my first idea is that this tool should download the latest current version of each file; but maybe you could add to the manifest a ‘latest beta’ parameter, and the tool would have two modes: ‘alpha tester’ that always uses latest versions, and ‘beta tester’ that uses the ‘latest beta’ version recorded. that way, the developer only has to update this parameter on the manifest to ‘beta release’ his code, while still banging code away, and collaborators stay in the bleeding edge.

[quote=“guessed, post:17, topic:165656”]b) Is the Wiki formatting engine consistent between wiki.micasaverde.com and code.mios.com?
I have a bunch to transcribe, so it’ll help to know the pitfalls in advance.[/quote]

no, it’s not.

[quote=“guessed, post:17, topic:165656”]c) Failing (b), do you have a template for homepages?
These have a tendancy to get all over the map (like all Wikis) without some structure being put in place. You can see the type of thing I’m referring to in my examples on wiki.micasaverde.com (for Paradox Security, Weather etc)
A template helps consumers “see” a consistent version of the world, even if there are different people producing the Plugin, and should ease the uptake.[/quote]

hum… i’m not the best designer out there; but i think the “Generic Lua Utilities” is a reasonable example.

19

Take a look at this for an idea of major “sections” for a template:

 http://wiki.micasaverde.com/index.php/Paradox_Alarm_Plugin

The sections can be broken out to sub-pages (or whatever), but the outline includes a general discussion, installation instructions, config instructions, technical details, pointers to Forum & Download along with “References” for any reference materials.

20

@Javier,
Looks like something has worked out how to “spam” the Ticket area of code.mios.com:

http://code.mios.com/trac/mios_ipserial/ticket/2

can you lock this down to prevent this from occurring?