Carpentry@UiO Merchandise

In order to promote Carpentry@UiO better than we do currently, I think we could use our logo and order a bag of stickers and a roll-up poster to be put outside the door of our workshop rooms. This roll-up should contain some basic info, i.e. where to find upcoming courses and how to join the community – information that doesn't quickly get old.
I think these promotional materials could be paid for by the library.
What do you think?

I think this is a brilliant idea!

Me too!

Looks like this got lost along the way. I will make another attempt at a budget and practicalities.
Due: post-summer Board meeting 2020

Looks like this has to be postponed further given the challenges around COVID-19. The communication team at UB is focusing on other, more pressing issues.
I propose the Board formulates a formal request for a couple of merch-items that should be in place by spring 2021, i.e. roll-up posters, stickers, perhaps t-shirts or buttons.

I will order two Beach Flags with our logo on them. Additionally, we will produce a number of buttons. I will also order stickers with our logo and investigate if we can get a bit of Carpentries merchandise from the global community. If not, I will also order an amount for distribution at on-site workshops.


Culture Invariant Tests for SQL

Why make this change?
It addresses issue #2319. Helps ensure that there are SQL tests that mirror the Cosmos tests that were added before.
What is this change?
Added test cases for SQL to make sure datatype conversion is CultureInfo.InvariantCulture
How was this tested?


Add note attack/release events

expected:
ToneSequencer.OnNoteAttack.AddListener(HandleOnNoteAttack);
ToneSequencer.OnNoteRelease.AddListener(HandleOnNoteRelease);

void HandleOnNoteAttack(int  chanel,  ToneNote note){
 // logic here
}

void HandleOnNoteRelease(int  chanel,  ToneNote note){
 // logic here
}

ToneSynth.OnNoteAttack.AddListener(HandleOnNoteAttack);
ToneSynth.OnNoteRelease.AddListener(HandleOnNoteRelease);

void HandleOnNoteAttack(int  chanel,  ToneNote note){
 // logic here
}

void HandleOnNoteRelease(int  chanel,  ToneNote note){
 // logic here
}

:tada: This issue has been resolved in version 1.10.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


silently ignore tiles not within target vs hard failure

Right now, we silently ignore failures if a tile isn't within target:

conditional: https://github.com/mapbox/vtcomposite/blob/9f849616fc70bd678060144d73d4ae986ad1ffd9/src/vtcomposite.cpp#L104
error handling: https://github.com/mapbox/vtcomposite/blob/9f849616fc70bd678060144d73d4ae986ad1ffd9/src/vtcomposite.cpp#L158

Let's decide whether or not we want to continue silently ignoring these errors or throw these errors back to the user.
cc @artemp @mapsam

errors are now caught and returned: https://github.com/mapbox/vtcomposite/blob/7103231ae30cd5139399547c5d0945fd6e97b7cd/src/vtcomposite.cpp#L175
we should decide how we want to handle this in JS cc @mapsam

We now have hard failures now if a tile isn't within target, with a helpful error message! https://github.com/mapbox/vtcomposite/commit/8bad715b3a1ea7f342cd2e81aa6934f382c1c745


feature request: possibility to use hostgroups

Compareable to the ansible inventory files, it would be great to support external collections of hosts.
(in addition to "--node" a "--group " option)
Ideally bolt should support to load group definition by:

file
url
puppetdb

Groups might be based on facts of puppetdb or manual definitions (a simple file/datastream which defines hostnames in groups).

This is good input for BOLT-318. I think the initial attempt will be file-based. PuppetDB might be added later (although it can be handled now by stringing together puppetdb-cli and file-based input for --nodes).
What do you aim to accomplish with a URL?

Bolt 0.16.2 and later releases have:

introduced a file-based inventory with grouping that you can refer to from the CLI and within plans
added a bolt-inventory-pdb script to generate inventory files from a set of PQL queries
added the --query flag to directly fetch a list of nodes from PuppetDB.

We plan to continue to enhance this. I believe this initial request is satisfied. If you have additional requests please file them as more specific issues.


WIP: Changes related to 1.6.1 release

use site.yml from tendrl-ansible repo
remove tendrl.yml - replace it by tendrl_{pre,post}.yml playbooks and expect the usage of site.yml from tendrl-ansible

@mbukatov Good points! I'll have to review and simplify the whole installation process... Reconsider all the existing playbooks and clean up, rename and polish them...

@mbukatov addressed some of your notes. I've also renamed the main playbooks this way:

ci_default_pre_setup.yml - tasks required at the beginning of the installation (before gdeploy),
ci_default_pre_tendrl.yml - tasks required before tendrl-ansible site.yml execution (between gdeploy and tendrl-ansible site.yml),
ci_default_post_tendrl.yml - tasks required after tendrl-ansible site.yml execution.


Remove installation of bcrypt in Charm code

BCrypt is installed during the run of the charm, which is a bad practice.
https://github.com/canonical/dex-auth-operator/blob/041cabc7ab051b72b0f024fbfef3e77a32c14196/src/charm.py#L26
It is used to hash the password only. We can use other libraries to perform string hashing which does not require the installation of the additional package.
Additionally, packages installed this way are not scanned during the CVE scanning.

Added to be fixed.

Low priority.


Page lints

Signed-off-by: kagarmoe<EMAIL_ADDRESS>:nut_and_bolt: Description: What code changed, and why?
Fixed a few lint warnings, mostly to try out the label action.
:chains: Related Resources
:+1: Definition of Done
:athletic_shoe: How to Build and Test the Change
:white_check_mark: Checklist
All PRs must tick these:

[ ] I have read the CONTRIBUTING document.
[ ] All commits signed-off for the Developer Certification of Origin.

With occasional exceptions, all PRs from Progress employees must tick these:

[ ] Is the code clear? (complicated code or lots of comments--subdivide and use well-named methods, meaningful variable names, etc.)
[ ] Consistency checked? (user notifications, user prompts, visual patterns, code patterns, variable names)
[ ] Repeated code blocks eliminated? (adapt and reuse existing components, blocks, functions, etc.)
[ ] Spelling, grammar, typos checked? (at a minimum use make spell in any component directory)
[ ] Code well-formatted? (indents, line breaks, etc. improve rather than hinder readability)

All PRs from Progress employees should tick these if appropriate:

[ ] Tests added/updated? (all new code needs new tests)
[ ] Docs added/updated? (all customer-facing changes)

Please add a note next to any checkbox above if you are NOT ticking it.
:camera: Screenshots, if applicable

💥 It labeled itself.


Split @font-face declaration into a separate file

Since it would be in a separate file, you can choose not to @import it by hand and provide the declaration by yourself. Our use case is to drop EOT and SVG formats and put it in line with other @font-faces using mixins.

This repo is an automated conversion from upstream and does not accept CSS fixes, see README.


Improve organization & installation

@mjohnson541 Yes, sklearn is in the molecule environment. I squashed the relevant commits.


Use NODATA mask to avoid out of bounds extractions

The extraction of our models is done on the whole extent of the images without considering the NODATA. Frequently we get false positives - especially from water -  in the out of bounds area since the band used for extraction exploits the entire 0-255 value range without dedicating any slot to the NODATA. It would be preferable to mask the extractions outside the valid area of the image.

In the STAC storage structure that is used as input for the extraction, each "single-band COG" actually includes 2 bands - one for the actual 8-bit dataset and one for its mask.
The mask band is a 8-bit binary array with 0s and 255s

gdalinfo

Band 1 Block=512x512 Type=Byte, ColorInterp=Gray
Overviews: 11214x16710, 5607x8355, 2803x4177, 1401x2088, 700x1044, 350x522, 175x261
Mask Flags: PER_DATASET
Overviews of mask band: 11214x16710, 5607x8355, 2803x4177, 1401x2088, 700x1044, 350x522, 175x261

rasterio

import rasterio
src = rasterio.open("/path/to/data")
msk = src.read_masks(1)

My suggestion would be to append a step by the end of the inference process to apply the mask to the extraction raster to remove any out of bounds features.

Good idea I think that's important to crop the over-site prediction.


Antiforgery goes at the end of filters

Fixes aspnet/Security#1009. Since Antiforgery should be at the end of filters by default set its order to be a large number by default.

:up::date:

:up::date:

:up::date:


Any plans to release?

Any plans to release? Thanks

Hi, thank you for showing interest in our project! We will release the code soon!

Hey, I am also looking forward to try your project. Tentatively, by when are you releasing the code?

Hey, I am also looking forward to try your project. Tentatively, by when are you releasing the code?

We are waiting for the paper acceptance. We have intentionally left certain details missing in the Arxiv version and will release the code as the paper gets accpeted.


query for new collectives in the past 2 weeks and daily

With creators email addresses. So @nicobarretoOC can manually email them a first nudge.
Thank you for taking the time to report an issue 🙏
The easier it is for us to reproduce it, the faster we can solve it.
So please try to be as complete as possible when filing your issue.

URL:
Logged in as: (just provide your username or the url of your profile)
Error message if any:
Expected result:
Browser:
Bonus point if you can add a screenshot :-)
Thank you and have a wonderful day/evening!
(and sorry for inconvenience! We'll do our best to solve this ASAP)

Query to get this data in the future (change the date):
select
	c.id as "collectiveId",
	slug,
	c."createdAt" as "collectiveCreatedAt",
	"HostCollectiveId",
	tags,
	u."firstName" as firstname,
	u."lastName" as lastname,
	u.email as email
from "Collectives" as c
left join "Users" u on c."CreatedByUserId" = u.id
where
	type ilike 'collective'
	and c."deletedAt" is null
	and u."deletedAt" is null
	and c."createdAt" > '2017-11-15';

added to wiki here


Failure: Cannot execute az acr build --registry <> --image <> .

Describe the bug
I am trying to build my docker image and push it to azure registry
I was following this guide: https://circleci.com/blog/deploying-container-based-node-apps-to-azure/
Command Name
az acr build
Logs this first:

Then errors:
The command failed with an unexpected error. Here is the traceback:
'NoneType' object is not callable
Traceback (most recent call last):
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 231, in invoke
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 657, in execute
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 691, in _run_job
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 328, in __call__
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/acr/build.py", line 106, in acr_build
TypeError: 'NoneType' object is not callable

To Reproduce:

az acr build --registry {} --image {} {}

Expected Behavior

build and push image to acr registry

Environment Summary
Windows-10-10.0.19041-SP0
Python 3.8.9
Installer: MSI

azure-cli 2.26.0

route to service team

Apparently if we do not provide a platform_variant, it does not work any more (it worked in 2.25.0)

I'm sure this will be resolved soon, but if you need to workaround the issue here are the steps I took:
If you're on Windows, you can most likely just install the earlier release
For those like me who are using WSL or other *nix-based OS and are using apt as your package manager, here are the steps I took to downgrade:
# Use apt-cache to install different versions of a package

sudo apt-get update

# get the available versions using: apt-cache showpkg <package-name>
# install a specific version using: sudo apt-get install package=version -V

sudo apt-get install azure-cli=2.23.0-1~focal -V

More info: How to Install Specific Version of Package using apt-get (linoxide.com)

The issue has been fixed since 2.26.1


export field with type string return error

I have cue file test.cue:
address: string

And when I call this, I got an error:
$ cue export test.cue
json: error calling MarshalJSON for type cue.Value: json: error calling MarshalJSON for type cue.Value: cannot convert incomplete value "string" to JSON

Working as intended. Feedback welcome as in how tool or docs can be clearer.

This issue has been migrated to https://github.com/cue-lang/cue/issues/32.
For more details about CUE's migration to a new home, please see https://github.com/cue-lang/cue/issues/1078.


Change old Failed Opportunities to Closed Lost

Consider making a celery task:

Every day, load Opportunities that are StageName = Failed, and CloseDate older than 3 days.
Change the StageName to Closed Lost

Probably could make a similar task to the one we use to update ACH charges when the bank finishes them.
@celery.task()
def update_ach_charges():

    lock = Lock(key='update-ach-charges-lock')
    lock.acquire()

    log = Log()

    log.it('---Starting batch ach job...')
    log.it('---Checking for status changes on ACH charges...')

    three_days_ago = (datetime.now(tz=zone) - timedelta(days=3)).strftime("%Y-%m-%d")
    today = datetime.now(tz=zone).strftime("%Y-%m-%d")

    opportunities = Opportunity.list(begin=three_days_ago, end=today, stage_name="ACH Pending")

    for opportunity in opportunities:
        if not opportunity.stripe_customer_id:
            continue
        amount = amount_to_charge(opportunity)
        log.it(
            f"---- ACH Charging ${amount} to {opportunity.stripe_customer_id} ({opportunity.name})"
        )
        try:
            charge(opportunity)
        except ChargeException:
            # TODO should we alert slack? Did not because we had no notifications here before
            pass

    log.send()

    lock.release()

Fixed in 706d42b855957ff172cdcf0f801f2850643c9566


Frontend: Beautify frontend UI

Remaining Things not done:

Cant really see the background issue on macbook might need to double check on @ekweirui windows to validate and try fix.

Things done:

Update table for better UI
Snaps matching options to top when screen is small

Fix fixed position profile button (see how when scroll down not over there)

snapping/resizing & top-right user icon looks great !
hmm on my windows machine the overflow is still there, will help to fix

Minor issue where the snapping of matching options to adapt to screen size causes question table to refresh. Backend is called  every time this happens. Is it possible to make this only happen once(when the page is first loaded)?

@thedesalizes The reason why this happen is caused useEffect was used at the QuestionPageView to rerender the question table in a certain format. I am able to let it refresh only once. However, it does not conform to what I have in mind... But I think it still looks okay so i have editted it as such :)


Set UserAgent in Scollector

In Bosun we set the UserAgent to Bosun/Version at
https://github.com/bosun-monitor/bosun/blob/c5de6525d157409de0e4013f6a58da1790d51192/cmd/bosun/main.go#L50
We should do the same for Scollector to help identify HTTP requests coming from Scollector

When we do this we should also allow the user to append to the UA, so external pollers can identify the org doing the polling if desired.


Cannot access parent component element

Hi, it's working on my pages component, but how can i call parent component element like header or sidebar ? i tried to call parent element id but that step it's not working.

Hi @martiendt,
I'm not sure I understand your problem very well. Could you be more precise please? What are you trying to achieve exactly?

Closing this issue as we had no answer in while. Feel free to re-open the issue if you still need help!


Could you please add support to @Observable macro

Hey @dmardashev,
I apologise for responding so late, but could you elaborate a little more on this topic? What exactly would you like to achieve? I would be very grateful for a piece of code that would clarify this topic for me
Once again, I'm sending you my apologies and have a good day,
Tomasz


Bunddel  Installation

unfortunately , an unexpected error, and blunder cannot continue

We only accept bug reports and pull requests on GitHub. StackOverflow may helps you.


Indexing OAI-PMH non-compliant repositories

So, BASE (http://www.base-search.net) indexes many repositories that have an OAI-PMH interface, which is already very useful and covers most of the respectable repositories.
But unfortunately, many repositories do not provide such an interface. And even when they provide one, the protocol is far from perfect: for instance, it is hard to detect accurately whether a full text is freely available behind a metadata record. Reaching out to the repository admins to encourage them to expose their metadata correctly is, from my experience, not effective at all.
If we want to go beyond this, I think we need to crawl!

One (conceptually) simple option would be to crawl for PDF files, extract metadata from them and dump this in a proper database with indexes. I think this is what CiteSeerX does, but only for papers within a particular field. In my experience the metadata that comes out of this is quite noisy.
The other option we have discussed would be to leverage existing scrapers (Zotero) to extract cleaner metadata from HTML pages. Zotero does the scraping pretty well, but I have no clue what crawling software we should use. Any idea? The scrapy framework looks nice but I'm a complete newcommer in this field so I have probably missed better options.

How much resources (servers) do we need for this? Where could we get them?

@wetneb thanks for this excellent summary. We can apply some pressure and communicate the importance of this initiative to the Internet Archive, they may be able to provide us with resources (disk/storage space). I can talk to Brewster this Friday about that possibility (a researcher VM with ~5tb of space, for starters). At very least, hopefully it will help us develop crawlers, write tutorials for the community to follow, and store documents temporarily (we can also use the openjournal user account to upload files directly to the Internet Archive, and they have an S3 style API for bulk upload. Internet Archive also does OCR on papers/pdfs that are uploaded, which can be a big mutual win. Re: noisy data, I am not sure how accurate their OCR is for academic (especially math-heavy) works. Perhaps worth exploring as an experiment.
Regarding crawlers, perhaps we can create a document an awesome-list of all crawling initiatives for academic papers, journals, databases, and metadata. I have ideas on how we can leverage libraries like scrapy -- as you elude to, I'm sure there existing tools (and I imagine creating n source-specific crawlers and reverse-engineering their indexing scheme, and having/maintaining a repository of these crawlers of each source the public to use, will be more success / more comprehensive than a deep crawl)
@pietsch, this is a fairly comprehensive list of publicly accessible sources that use OAI-PMH, right?

Please see comments below.
It is important not to reinvent wheels. Some of what we discuss
falls into that domain.
On 11/30/15 5:45 PM, Antonin wrote:

So, BASE (http://www.base-search.net) indexes many repositories that have an OAI-PMH interface, which is already very useful and covers most of the respectable repositories.
But unfortunately, many repositories do not provide such an interface. And even when they provide one, the protocol is far from perfect: for instance, it is hard to detect accurately whether a full text is freely available behind a metadata record. Reaching out to the repository admins to encourage them to expose their metadata correctly is, from my experience, not effective at all.
If we want to go beyond this, I think we need to crawl!

One (conceptually) simple option would be to crawl for PDF files, extract metadata from them and dump this in a proper database with indexes. I think this is what CiteSeerX does, but only for papers within a particular field. In my experience the metadata that comes out of this is quite noisy.
No longer, CiteSeerX now crawls for all scholarly documents.
Please note that some repositories prevent crawling with their
robots.txt, except for Googlebot.

Yes, it is noisy but not that bad and is always getting better. Others
are doing this and
contributing to the extraction algorithms. Please see our previous list
of extraction methods.
We may have a tutorial at major conference soon.

The other option we have discussed would be to leverage existing scrapers (Zotero) to extract cleaner metadata from HTML pages. Zotero does the scraping pretty well, but I have no clue what crawling software we should use. Any idea? The scrapy framework looks nice but I'm a complete newcommer in this field so I have probably missed better options.
Our crawling code is available on the CiteSeerX GitHub. We are always
crawling and recently with
Semantic Scholar at AI2. We use Heritrix, an excellent tool.
How much resources (servers) do we need for this? Where could we get them?
This is storage and bandwidth intensive. The problems are:

crawling is very time consuming and needs a great deal of coordinated
parallel threads.

crawling brings back unwanted PDFs which must be filtered or
classified. We have
a few papers on this if anyone is interested.

a scholarly paper PDF is about one Meg - a million a Tera. However,
the others have
to are usually stored.

We've found a factor of 3 for all documents we expect to crawl. That is
we crawl 3 times
as many PDFs for the same number of PDF files.
We share our crawl seeds if anyone wants to use them.
Who here crawls?

Reply to this email directly or view it on GitHub:
https://github.com/OpenJournal/central/issues/8

These papers may be of help. One can publish improvements. We would very
much like to improve ours. Suggestions most welcomed.
Best
Lee
On 11/30/15 7:04 PM, Michael E. Karpeles wrote:

@wetneb thanks for this excellent summary. We can apply some pressure and communicate the importance of this initiative to the Internet Archive, they may be able to provide us with resources (disk/storage space). I can talk to Brewster this Friday about that possibility (a researcher VM with ~5tb of space, for starters). At very least, hopefully it will help us develop crawlers, write tutorials for the community to follow, and store documents temporarily (we can also use the openjournal user account to upload files directly to the Internet Archive, and they have an S3 style API for bulk upload. Internet Archive also does OCR on papers/pdfs that are uploaded, which can be a big mutual win. Re: noisy data, I am not sure how accurate their OCR is for academic (especially math-heavy) works. Perhaps worth exploring as an experiment.
Regarding crawlers, perhaps we can create a document an awesome-list of all crawling initiatives for academic papers, journals, databases, and metadata. I have ideas on how we can leverage libraries like scrapy -- as you elude to, I'm sure there existing tools (and I imagine creating n source-specific crawlers and reverse-engineering their indexing scheme, and having/maintaining a repository of these crawlers of each source the public to use, will be more success / more comprehensive than a deep crawl)
@pietsch, this is a fairly comprehensive list of publicly accessible sources that use OAI-PMH, right?

Reply to this email directly or view it on GitHub:
https://github.com/OpenJournal/central/issues/8#issuecomment-160803883

@cleegiles Thanks for the excellent contribution. Also, for the record, I view OpenJournal as a working group for reducing (exactly as you say) re-implemented wheels. So I'm on board! I don't anticipate building anything directly through OpenJournal -- if I contribute to something, it will be an existing project (unless there's a very compelling reason why something new needs to be built, and if that's the case, I'd still prefer someone else leads it, i.e. limited bandwidth).
There are several folks affiliated w/ the Internet Archive I know who are working on crawling efforts. Some aren't comfortable announcing. @nthmost can likely share her ideas for pubmed crawling. I have experience crawling, am happy to participate, but do not currently work on a crawler.
@cleegiles I'll try to generate some traffic to this thread by pinging other institutions and seeing if they can weigh in on the status of their crawlers -- thanks for nudging us in that direction.

@cleegiles Thanks for the clarification, it's great that CiteSeerX is not limited to comp. sci. anymore!
Do not get me wrong, the metadata CiteSeerX extracts is very useful. But at http://dissem.in we need to match publications with preprints, which is quite hard as soon as the title or the authors differ by a few words. So, metadata quality is critical for us.
I can imagine that CiteSeerX requires a lot of resources indeed. We cannot afford this for dissemin, and this is the reason why I thought using scrapers would be cheaper (both in terms of bandwidth, storage, and computing) and could potentially yield cleaner metadata. But this option only works for repositories, not home pages.
I have been playing around on indexing researchgate.net recently (with crawling and scraping), and I have been in touch with Mike Taylor who has started something along these lines for SSRN.

And even when they provide one, the protocol is far from perfect: for instance, it is hard to detect accurately whether a full text is freely available behind a metadata record.

Yes, this is an issue we're trying to deal with in ipfs/archives#3 (enriching OAI-PMH metadata with fulltext links).
@cleegiles Is this something that CiteSeerX could help with?

@davidar that's awesome! I hope you will succeed.

@wetneb well, my plan was basically the same as what you outlined in the OP, so I'm afraid I might be reinventing the wheel? Perhaps this is something we could collaborate on?

@davidar I would love to! Joining the discussion there then.

Hi @mekarpeles,

@pietsch, this is a fairly comprehensive list of publicly accessible sources that use OAI-PMH, right?

Yes, these are the 3881 OAI-PMH sources BASE is currently harvesting (in intervals). The other lists I am aware of are the Directory of Open Access Journals (DOAJ) and the official (if incomplete and out-of-date) list of OAI-PMH repositories.

We do this with our crawler. If the link goes directly to a non-open
source publisher, there is no reason to
crawl. We have a blacklist and whitelist of where we go now which we can
share. It's fairly complete.
Another way is to parse the link to the document. Links that are not
directly to pdfs are usually not
downloadable. It would be useful to do a sample to see how often this is
true, but we've found it
to nearly always be the case. A counter example is where one has to sign
in but anyone can have
an account.
On 12/1/15 3:03 AM, David A Roberts wrote:

And even when they provide one, the protocol is far from perfect: for instance, it is hard to detect accurately whether a full text is freely available behind a metadata record.
Yes, this is an issue we're trying to deal with in ipfs/archives#3 (enriching OAI-PMH metadata with fulltext links).

@cleegiles Is this something that CiteSeerX could help with?

Reply to this email directly or view it on GitHub:
https://github.com/OpenJournal/central/issues/8#issuecomment-160888712

The best metadata seems to be the Web of Science (WoS) but it has to be
purchased and is not cheap.
This seems to be a good source of metadata, but we have not compared it
to ours

http://research.microsoft.com/en-us/projects/mag/
We currently have a project to clean up our and other metadata.

On 12/1/15 2:27 AM, Antonin wrote:

@cleegiles Thanks for the clarification, it's great that CiteSeerX is not limited to comp. sci. anymore!
Do not get me wrong, the metadata CiteSeerX extracts is very useful. But at http://dissem.in we need to match publications with preprints, which is quite hard as soon as the title or the authors differ by a few words. So, metadata quality is critical for us.
I can imagine that CiteSeerX requires a lot of resources indeed. We cannot afford this for dissemin, and this is the reason why I thought using scrapers would be cheaper (both in terms of bandwidth, storage, and computing) and could potentially yield cleaner metadata. But this option only works for repositories, not home pages.
I have been playing around on indexing researchgate.net recently (with crawling and scraping), and I have been in touch with Mike Taylor who has started something along these lines for SSRN.

Reply to this email directly or view it on GitHub:
https://github.com/OpenJournal/central/issues/8#issuecomment-160882106

Hi everybody,
As @mekarpeles mentioned, I've been working on PubMed collection efforts for over a year now.  That code is represented in the metapub project and install-able via pypi (pip install metapub).
The primary purpose of the FindIt tool within metapub is to be able to pull fulltext article matter (just PDFs right now) at high identity confidence.  I.e. if a researcher thinks they are getting pubmed ID #123456, the result of using FindIt should be exactly that article about 99% of the time.
Here's the overview of how it works.  Starting from a PubMed ID, FindIt does the following steps on each article:

uses the pubmed ID to pull down the PubMed XML for the article
uses the PubMedCentral ID, if any, to produce a url to a pdf
if not in PubMedCentral, looks up the journal name within the FindIt machinery to see if we can apply a known “dance” to get a PDF link on the publisher’s website.
if journal name not currently filed in FindIt, reports as “NOFORMAT"

I can explain all of this in detail -- for now I think it suffices to say that my approach is very different from crawling or screen scraping, and this is very much by design.
I built and deployed this engine in production at a genetic testing/diagnostic company to save scientists time in tracking down the article texts they needed to research and prove the calls they were making on people's genetic test reports.
As a result, FindIt's coverage of the NCBI journal list is heavily skewed towards medical genetics, and its testing has focused on pubmed citations found in HGMD and Clinvar.  This gave FindIt a nicely controlled constraint for its success; now it's time to branch out and try to complete its coverage across all PubMed domains.
I recently completed a long-running coverage test in which I iterated over every named NCBI journal from the Entrez list, found 3 to 5 article IDs per journal from different years (if possible), and then ran FindIt over those IDs.  (Total pmids = ~117k.)  I have yet to analyze these results, but will probably do so on the plane back from Hawai'i (where i've been hiding during the evolution of all this discussion).
After completing metapub's coverage of PubMed, I'm interested in starting a project with the same design constraints (i.e. high confidence, next-to-no actual scraping) that covers all journals that have DOIs.  There is a lot of machinery in metapub that could apply to a broader swath of disciplines via the use of the CrossRef API and the dx.doi.org redirect.
I'm eager to get more involved with all of you!

Good thing to do.
If I understand what you are doing, you are interested in full
documents, PDFs?
If so, how do you extract the text from the PDF?
Best
Lee
On 12/1/15 4:16 PM, nthmost wrote:

Hi everybody,
As @mekarpeles mentioned, I've been working on PubMed collection efforts for over a year now.  That code is represented in the metapub project and install-able via pypi (pip install metapub).
The primary purpose of the FindIt tool within metapub is to be able to pull fulltext article matter (just PDFs right now) at high identity confidence.  I.e. if a researcher thinks they are getting pubmed ID #123456, the result of using FindIt should be exactly that article about 99% of the time.
Here's the overview of how it works.  Starting from a PubMed ID, FindIt does the following steps on each article:

uses the pubmed ID to pull down the PubMed XML for the article
uses the PubMedCentral ID, if any, to produce a url to a pdf
if not in PubMedCentral, looks up the journal name within the FindIt machinery to see if we can apply a known “dance” to get a PDF link on the publisher’s website.
if journal name not currently filed in FindIt, reports as “NOFORMAT"

I can explain all of this in detail -- for now I think it suffices to say that my approach is very different from crawling or screen scraping, and this is very much by design.
I built and deployed this engine in production at a genetic testing/diagnostic company to save scientists time in tracking down the article texts they needed to research and prove the calls they were making on people's genetic test reports.
As a result, FindIt's coverage of the NCBI journal list is heavily skewed towards medical genetics, and its testing has focused on pubmed citations found in HGMD and Clinvar.  This gave FindIt a nicely controlled constraint for its success; now it's time to branch out and try to complete its coverage across all PubMed domains.
I recently completed a long-running coverage test in which I iterated over every named NCBI journal from the Entrez list, found 3 to 5 article IDs per journal from different years (if possible), and then ran FindIt over those IDs.  (Total pmids = ~117k.)  I have yet to analyze these results, but will probably do so on the plane back from Hawai'i (where i've been hiding during the evolution of all this discussion).
After completing metapub's coverage of PubMed, I'm interested in starting a project with the same design constraints (i.e. high confidence, next-to-no actual scraping) that covers all journals that have DOIs.  There is a lot of machinery in metapub that could apply to a broader swath of disciplines via the use of the CrossRef API and the dx.doi.org redirect.
I'm eager to get more involved with all of you!

Reply to this email directly or view it on GitHub:
https://github.com/OpenJournal/central/issues/8#issuecomment-161098227

@cleegiles as an aside, the Internet Archive does OCR on any pdfs uploaded to them (I think this was @nthmost's plan, however I'm interested in hear your opinions). I'm sure many institutions would benefit from more contributions towards a more proficient library for extracting text from pdf.
Ideally, in the future, the .tex version of the paper will be available via something like github... But as frustrating as that is, I'll keep it contained as a separate issue :)

Ideally, in the future, the .tex version of the paper will be available via something like github...

That would be amazing, unfortunately arXiv seems to be the only ones distributing TeX sources currently

Love what's going on in this thread
@davidar can't wait for a full offline-friendly experience of arxiv with ipfs+TeX.js :)

PSF to text extraction for "quality" information is still an open
question. I would guess
that IA is using PDFBox, a reasonable selection.
It's very important to know what tool that is being used for PDF to text
extraction.
There are many available. There are companies that make a living on
their modifications
of existing software or creating their own, i.e. gonitro.com. I would
put their proprietary software as state of
the art in comparison with Google. Many scientists are very concerned
how data can be extracted from PDFs since this is
the only place that some data exists and can be digitized - odd isn't it.
Most open source extractors are ok but not of high quality, this is one
reason CiteSeerX's extraction flaws.
We use either PDFBox or PDFlib TET (not open source). Google's is
extremely good! Not released yet.
AI2 I've been told is about to release a very good converter. We haven't
seen it, but many of
the tools they've released so far have been excellent - we use them.
Interesting, surprisingly few scholarly papers are published in .tex,
many are in .doc or .docx,
especially in medicine, engineering or sciences outside of computer
science and physics.
On 12/1/15 7:24 PM, Michael E. Karpeles wrote:

@cleegiles as an aside, the Internet Archive does OCR on any pdfs uploaded to them (I think this was @nthmost's plan, however I'm interested in hear your opinions). I'm sure many institutions would benefit from more contributions towards a more proficient library for extracting text from pdf.
Ideally, in the future, the .tex version of the paper will be available via something like github... But as frustrating as that is, I'll keep it contained as a separate issue :)

Reply to this email directly or view it on GitHub:
https://github.com/OpenJournal/central/issues/8#issuecomment-161140126

@cleegiles my work's not focused on doing OCR, as the PDF format vis-a-vis people's usage of it in academic papers is pretty far removed from standardized; I'd rather do as @mekarpeles has suggested and upload PDFs to the Archive to be OCRed there.
That said, at my last job, we were able to use pdfminer (a Python library) to good effect to turn medical genetics papers (in English) into machine-indexable text.  We built indexes over these texts and mapped mentions of important genetics concepts back to their pubmed IDs, so that medical concepts (referenced in the NIH medgen database) could be mapped to pubmed citation evidence.

Sorry for being 6+ years late to the party. ;)

Reaching out to the repository admins to encourage them to expose their metadata correctly

COAR focuses on this part of the puzzle quite a bit. They have various ongoing initiatives like https://www.coar-repositories.org/news-updates/ccsd-and-coar-announce-plans-to-launch-preprint-directory/ .

extract cleaner metadata from HTML pages

https://scholar.archive.org/ is arguably doing just this: all the PDFs and other full text sources the Internet Archive finds with its scanning and crawling efforts get mined for academic works to index. Everyone go contribute! ;) https://github.com/internetarchive/fatcat

@nemobis +1! And to everyone else who continue to tireless further the space. I know @bnewbold et al have leveraged the amazing work of others in the community to build factact into another great resource in the space. Proud to watch these efforts mature and grateful for everyone's work.


I Did Everything Just Like The Video, and there is an error code

This Serverless Function has crashed.
Your connection is working correctly.
Vercel is working correctly.

read the last line on that page 🧠


How to use the low-level callbacks (or cast *00 objects to their proxy counterparts)

Hi there,
First of all, I enjoy this library quite a bit. I wanted to make a ListView using a TreeListModel. As mentioned in #142 and #146, callbacks like TreeListModelCreateModelFunc (the callback type used to create child ListModels), are still quite low level. I know that, somewhere in the future, it would be nice to have better callbacks, but for now I'm trying to figure out how to deal with them. As you have explained, the *00 objects are lower level that the regular objects, which are proxies. How do I get a regular object from a *00 object? At least, that's what I think I want. I'm not quite sure, because I don't understand it very well yet.

Have I seen your name before? I can not really remember. Can I be sure that you are not from Russia?
The ListView and related GTK4 widgets are not that easy, I have seen related questions for C, Rust and Python in the Gnome forum, without really convincing answers. But the old  GtkTreeView widgets have been complicated too, I used that in 2007-2009 from Ruby, following the GTK2 book of A. Krause. I think I have collected all my current knowledge in https://ssalewski.de/gtkprogramming.html#_listview_columnview_gridview_and_dropdown already.
The best starting point is generally a plain working C program. So you should try to find one, or create it yourself. Python, Vala or another language would be OK also, but the program should have not more than maybe 300 lines of code at least. Some of the gitlab provided examples are unfortunately very large and very complicated. When we have a working C example, then translating that to Nim is in most cases not that difficult, but we may have to create Nim macros in some cases. I am not really good in Nim macros, do you have already a lot of experience with it?
For the relation of the proxy objects and the 00 low level types: You will find a few manually created examples in the gen.nim file, for example
proc newFileChooserDialog*(title: cstring = nil; parent: Window = nil; action: FileChooserAction): FileChooserDialog =
  let gobj = gtk_file_chooser_dialog_new(title, if parent.isNil: nil else: cast[ptr Window00](parent.impl), action)
  if g_object_get_qdata(gobj, Quark) != nil:
    result = cast[type(result)](g_object_get_qdata(gobj, Quark))
    assert(result.impl == gobj)
  else:
    fnew(result, $1.finalizeGObject)
    result.impl = gobj
    GC_ref(result)
    discard g_object_ref_sink(result.impl)
    g_object_add_toggle_ref(result.impl, toggleNotify, addr(result[]))
    g_object_unref(result.impl)
    assert(g_object_get_qdata(result.impl, Quark) == nil)
    g_object_set_qdata(result.impl, Quark, addr(result[]))

That is an arbitrary example, maybe a bad one, but there are a few more. Most allocator procs, like newButton() are generated automatically, you may find them in gtk4.nim and follow that shape when you have to create allocator procs manually. Basically, the impl field is the reference to the low level object. But all that is a bit complicated, as the Nim memory management system has to handle all that, and as low level widgets may temporary exists without a proxy object. This may occur for example, when we add a widget to a GTK container like a box, then forget about it, and later call getChild() to get it back. For this we use the GTK toggleref formalism. Took me about 100 hours to get that working, I can not remember details, but now I generally only follow the existing examples when I have to create more manually. For testing, I compile then with -d:gintroDebug and run a test program, when I then see destroy() calls, I can assume that memory management works. Testing with valgrind is also an option, but GTK internally does not free all resources, so Valgrind generally complains about each GTK program, even about C programs from C GTK devs. For testing the --mm:refc version, I generally add GC_Fullcollect() calls, that may not really proof that all resources get freed, but at least that should not crash.
So please provide a small working test program, preferable in C or Python, and I can tell you in more detail how to proceed.

Thank you for your quick response. I have made an earlier issue regarding compilation a few days ago. I am Dutch ;). I have used TreeListModels before, in the rust programming language, so I think that I can write up a little example project in C, that uses them. I'll get back to you about that.
Thank you for the example and explanation. I'll look through the gen.nim file for relevant examples!

I am Dutch

Great! A working example in C or Rust would be a good starting point.

I did a short Google search about use of the  TreeListModel and only found the old
https://blog.gtk.org/2020/06/08/more-on-lists-in-gtk-4/
which is not easy to understand. The referenced listview settings app from gtk4-demo shows us at least two problems:

It uses C macros like G_DECLARE_FINAL_TYPE() to create new GObjects with its own GObject properties. Creating new GObjects is not yet supported by the gintro bindings. Would be nice to have support for that, but we would have to convert all the C macros to Nim code, which is not easy. I tried it seven years ago with the oldgtk3 bindings for my old NEd editor. I would guess that supporting this may took more than 100 hours, maybe more likely 200 hours for me, as I do not lnow much about all the GObject internals, and as I am not very experienced with Nim macros.

The ui file contains segments like handler="search_changed" which directly calls a C function from the UI file. This is not really supported by gintro. Well, we may create a proc with cdecl pragma, and low level 00 parameters. With the right compiler and linker options, we may get that proc called.

All that is difficult, and there may be more issues. Maybe we could work around somehow. For Nim all that is possible somehow, by use of a lot of low level stuff with pointers. But finally we would like to have some clean, typesafe interface, and that can be hard.
Actually, my feeling is, that GTK4 is even more C centric, supporting language bindings becomes harder and harder. I heard that some stuff has even no Python support.

hi there. I have written a simple <100 lines example of what I'm trying to do:
`#include <gtk/gtk.h>
#include <gio/gio.h>
GListModel *create_child_lists(GObject *item, gpointer ignore)
{
char *array[] = {"test1", "test2", NULL};
GtkStringList *list = gtk_string_list_new((const char * const *) array);
return G_LIST_MODEL(list);
}
void setup(GtkSignalListItemFactory *self, GObject *object, gpointer ignore)
{
GtkListItem *item = GTK_LIST_ITEM(object);
GtkTreeExpander *expnr = gtk_tree_expander_new();
GtkLabel *label = gtk_label_new("");
gtk_tree_expander_set_child(expnr, label);
gtk_list_item_set_child(item, expnr);
}
void bind(GtkSignalListItemFactory *self, GObject *object, gpointer ignore)
{
GtkListItem *item = GTK_LIST_ITEM(object);
gpointer *real_item = gtk_list_item_get_item(item);
GtkTreeListRow *row = GTK_TREE_LIST_ROW(real_item);
GtkTreeExpander *expnr = gtk_list_item_get_child(item);
gtk_tree_expander_set_list_row(expnr, row);
GtkLabel label = gtk_tree_expander_get_child(expnr);
GtkStringObject string_object = gtk_tree_list_row_get_item(row);
const char *string = gtk_string_object_get_string(string_object);
gtk_label_set_label(label, string);
}
GtkListView *get_tree_view()
{
char *array[] = {"test1", "test2", NULL};
GtkStringList *list = gtk_string_list_new((const char * const *) array);
GtkTreeListModel *treeList = gtk_tree_list_model_new(G_LIST_MODEL(list), false, false, create_child_lists, NULL, NULL);
GtkSelectionModel *selectionModel = gtk_single_selection_new(G_LIST_MODEL(treeList));
GtkSignalListItemFactory *factory = gtk_signal_list_item_factory_new();
g_signal_connect(factory, "setup", setup, NULL);
g_signal_connect(factory, "bind", bind, NULL);

GtkListView *view = gtk_list_view_new(selectionModel, GTK_LIST_ITEM_FACTORY(factory));
return view;

}
static void activate(GtkApplication *app, gpointer user_data)
{
GtkWindow *window;
window = GTK_WINDOW(gtk_application_window_new(app));
gtk_window_set_title(GTK_WINDOW(window), "Window");
gtk_window_set_default_size(GTK_WINDOW(window), 200, 200);
gtk_window_set_child(window, GTK_WIDGET(get_tree_view()));
gtk_widget_show(window);

}
int main(int argc, char **argv)
{
GtkApplication *app;
int status;
app = gtk_application_new("org.gtk.example", G_APPLICATION_DEFAULT_FLAGS);
g_signal_connect(app, "activate", G_CALLBACK(activate), NULL);
status = g_application_run(G_APPLICATION(app), argc, argv);
g_object_unref(app);

return status;

}`

hm, perhaps I shall try to tackle gobject sub-classing and binding to those macros

Great, looks fine. I assume I can compile and run it? I will try to convert it to Nim this evening or tomorrow. Is there no GtkTreeListModelCreateModelFunc involved? I think for that beast we would need a Nim macro?
Yesterday I found this description: https://stackoverflow.com/questions/74556059/how-to-build-a-tree-in-gtk4-4-10
But I have not followed it yet. Well, I will try your C code and convert it to Nim, maybe to ugly Nim first.

Yes, compiles and runs, but with many compiler warnings:
gcc t.c -o t `pkg-config --cflags --libs gtk4`
In file included from /usr/include/gtk-4.0/gtk/gtkapplication.h:27,
                 from /usr/include/gtk-4.0/gtk/gtkwindow.h:33,
                 from /usr/include/gtk-4.0/gtk/gtkaboutdialog.h:30,
                 from /usr/include/gtk-4.0/gtk/gtk.h:34,
                 from t.c:2:
t.c: In function ‘bind’:
/usr/include/gtk-4.0/gtk/gtkwidget.h:46:43: warning: initialization of ‘GtkTreeExpander *’ {aka ‘struct _GtkTreeExpander *’} from incompatible pointer type ‘GtkWidget *’ {aka ‘struct _GtkWidget *’} [-Wincompatible-pointer-types]
   46 | #define GTK_WIDGET(widget)                (G_TYPE_CHECK_INSTANCE_CAST ((widget), GTK_TYPE_WIDGET, GtkWidget))
      |                                           ^
t.c:26:30: note: in expansion of macro ‘GTK_WIDGET’
   26 |     GtkTreeExpander *expnr = GTK_WIDGET(gtk_list_item_get_child(item));
      |                              ^~~~~~~~~~
t.c: In function ‘get_tree_view’:
t.c:38:92: warning: passing argument 4 of ‘gtk_tree_list_model_new’ from incompatible pointer type [-Wincompatible-pointer-types]
   38 |     GtkTreeListModel *treeList = gtk_tree_list_model_new(G_LIST_MODEL(list), false, false, create_child_lists, NULL, NULL);
      |                                                                                ^~~~~~~~~~~~~~~~~~
      |                                                                                |
      |                                                                                GListModel * (*)(GObject *, void *) {aka struct _GListModel * (*)(struct _GObject *, void *)}
In file included from /usr/include/gtk-4.0/gtk/gtktreeexpander.h:27,
                 from /usr/include/gtk-4.0/gtk/gtk.h:266:
/usr/include/gtk-4.0/gtk/gtktreelistmodel.h:64:98: note: expected ‘GtkTreeListModelCreateModelFunc’ {aka ‘struct _GListModel * (*)(void *, void *)’} but argument is of type ‘GListModel * (*)(GObject *, void *)’ {aka ‘struct _GListModel * (*)(struct _GObject *, void *)’}
   64 |                                                                  GtkTreeListModelCreateModelFunc create_func,
      |                                                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~
In file included from /usr/include/glib-2.0/gobject/gobject.h:30,
                 from /usr/include/glib-2.0/gobject/gbinding.h:31,
                 from /usr/include/glib-2.0/glib-object.h:24,
                 from /usr/include/gtk-4.0/gtk/css/gtkcssenumtypes.h:11,
                 from /usr/include/gtk-4.0/gtk/css/gtkcss.h:34,
                 from /usr/include/gtk-4.0/gtk/gtk.h:30:
/usr/include/glib-2.0/gobject/gsignal.h:515:59: warning: passing argument 3 of ‘g_signal_connect_data’ from incompatible pointer type [-Wincompatible-pointer-types]
  515 |     g_signal_connect_data ((instance), (detailed_signal), (c_handler), (data), NULL, (GConnectFlags) 0)
      |                                                           ^~~~~~~~~~~
      |                                                           |
      |                                                           void (*)(GtkSignalListItemFactory *, GObject *, void *) {aka void (*)(struct _GtkSignalListItemFactory *, struct _GObject *, void *)}
t.c:42:5: note: in expansion of macro ‘g_signal_connect’
   42 |     g_signal_connect(factory, "setup", setup, NULL);
      |     ^~~~~~~~~~~~~~~~
/usr/include/glib-2.0/gobject/gsignal.h:411:67: note: expected ‘GCallback’ {aka ‘void (*)(void)’} but argument is of type ‘void (*)(GtkSignalListItemFactory *, GObject *, void *)’ {aka ‘void (*)(struct _GtkSignalListItemFactory *, struct _GObject *, void *)’}
  411 |                                                GCallback          c_handler,
      |                                                ~~~~~~~~~~~~~~~~~~~^~~~~~~~~
/usr/include/glib-2.0/gobject/gsignal.h:515:59: warning: passing argument 3 of ‘g_signal_connect_data’ from incompatible pointer type [-Wincompatible-pointer-types]
  515 |     g_signal_connect_data ((instance), (detailed_signal), (c_handler), (data), NULL, (GConnectFlags) 0)
      |                                                           ^~~~~~~~~~~
      |                                                           |
      |                                                           void (*)(GtkSignalListItemFactory *, GObject *, void *) {aka void (*)(struct _GtkSignalListItemFactory *, struct _GObject *, void *)}
t.c:43:5: note: in expansion of macro ‘g_signal_connect’
   43 |     g_signal_connect(factory, "bind", bind, NULL);
      |     ^~~~~~~~~~~~~~~~
/usr/include/glib-2.0/gobject/gsignal.h:411:67: note: expected ‘GCallback’ {aka ‘void (*)(void)’} but argument is of type ‘void (*)(GtkSignalListItemFactory *, GObject *, void *)’ {aka ‘void (*)(struct _GtkSignalListItemFactory *, struct _GObject *, void *)’}
  411 |                                                GCallback          c_handler,
      |                                                ~~~~~~~~~~~~~~~~~~~^~~~~~~~~
salewski@hx90 ~/hhh $ gcc -v
Using built-in specs.
 Seems to compile and run:
#[
GListModel *create_child_lists(GObject *item, gpointer ignore)
{
    char *array[] = {"test1", "test2", NULL};
    GtkStringList *list = gtk_string_list_new((const char * const *) array);
    return G_LIST_MODEL(list);
}
]#

import gintro/[gtk4, gobject, gio, glib]

# this is called from GTK, so all data types are the 00 onces.
proc create_child_lists(item: ptr gobject.Object00; ignore: pointer): ptr ListModel00 {.cdecl.} =
  let list = newStringList("test1", "test2")
  list.ignoreFinalizer = true # because GTK frees it itself
  return cast[ptr ListModel00](list.impl)

proc setup(self: SignalListItemFactory; obj: Object) =
  let item = cast[ListItem](obj)
  let expnr = newTreeExpander()
  let label = newLabel() # or ""
  expnr.setChild(label)
  item.setChild(expnr)

proc bnd(self: SignalListItemFactory; obj: Object) =
  let item = cast[ListItem](obj)
  let realItem = item.getItem
  let row = cast[TreeListRow](realItem)
  let expnr: TreeExpander =  cast[TreeExpander](item.getChild)
  expnr.setListRow(row)
  let label = cast[Label](expnr.getChild)
  let stringObject = cast[StringObject](row.getItem)
  let str = stringObject.getString
  label.setLabel(str)

proc getTreeView: ListView =
  let list = newStringList("test1", "test2")
  #proc newTreeListModel*(root: gio.ListModel; passthrough: bool; autoexpand: bool;
  #  createFunc: TreeListModelCreateModelFunc; userData: pointer; userDestroy: DestroyNotify): TreeListModel =
  let treeList = newTreeListModel(cast[ListModel](list), false, false, create_child_lists, nil, nil)
  let selectionModel = newSingleSelection(cast[ListModel](treeList))
  let factory = newSignalListItemFactory()
  factory.connect("setup", setup)
  factory.connect("bind", bnd)
  return newListView(selectionModel, factory)

proc activate(app: Application) =
  let window = newApplicationWindow(app)
  window.title = "Window"
  window.defaultSize = (200, 200)
  window.setChild(getTreeView())
  window.present # gtk_widget_show(GTK_WIDGET(window));

proc main =
  let app = newApplication("org.gtk.example")
  app.connect("activate", activate)
  let status = app.run
  quit(status)

main()

I am not very experienced in C, so I'm not so sure how to fix those warning. The GtkTreeListModelCreateModelFunc type id actually one of the warnings. You can cast the callback function to that type to get rif of that warning, but it works perfectly like this.
That was rather quick with the Nim version. Is there anything I could do? I might have a spare hour or two to work on it tomorrow.

Yes, was easy. Took me only one hour. You can test if this program shape can do really something useful for you. I guess we have somehow to connect callbacks to the items in the tree? The gitlab example seems to do that in the XML UI file? Later, we would have to create a macro that replaces the newTreeListModel() call. The macro has to create a low level function, and that created function would then call the high level user function, to avoid all the 00 types, the ignoreFinalizer= true and all that. The gintro package should have a few examples of such macros. We need such macros whenever we have to pass a function as a parameter to a GTK function. I think one example is the setDrawFunction for the GtkDrawingArea in GTK4, or the idleAdd() and timer functions.  Nims converters may be an option, but generally I avoid them.

Very well, I shall try to create a macro to replace newTreeListModel today.
As far as I'm aware, you don't need to connect callbacks to the items. They are just data. The widgets you use to display this data might use callback, but that will just use the regular way of doing it, so I don't think we'll have any work there.

create a macro to replace newTreeListModel today.

You may first try to create a nice macro from scratch. I heard that the book of Mr. Rumpf has some macro docs. If you don't have that book, you may consult the official tutorial part 3 (from A. Döhring), the Manning book or my book. My book is still freely available.
Generally you should try to create a macro which does notdo  too much string manipulations, but mostly work on the AST. My own macros use still a lot of string manipulation, so Rumpf told me once they are ugly. And I was unable go get any support from core devs for my macros. My mconnect() macro is indeed very ugly, but I have not the skills, time and motivation to rewrite it.
If "from scratch" is still too much for you, you may use the two macros from gimplgtk.nim as a starting point
grep macro gimplgtk.nim
# unset -- the macro can do this already
macro setCellDataFunc*(column: TreeViewColumn; renderer: CellRenderer; fn: untyped = nil; arg: typed = nil): untyped =
macro msetDrawFunc(da: DrawingArea; p: untyped; arg: typed; ignoreArg: bool): untyped =

Maybe a real universal macro would be possible, that is a macro that works whenever we have to pass a Nim high level function to a GTK function? But that is something which only real experts may be able to do.
Note that I am doing only minimal Nim and GTK work currently, so you should not expect much help from be. For the reasons see
https://forum.nim-lang.org/t/9753#64224

It's not nearly as frustrating as having to read your ramblings.

and for GTK gintro generally, my replies in
https://www.reddit.com/r/nim/comments/10at44v/why_does_the_example_gtk_application_not_compile/

Thank you for the reference material. I have forked the repo and am now working on this macro. Any particular reason to use macros? I feel like the same can be achieved with a regular proc.

I feel like the same can be achieved with a regular proc.

You may try it with a proc :-)
Note that the user should be free to pass an arbitrary named proc instead of our create_child_lists() from above, with arbitrary named parameters, and optionally one more parameter of arbitrary data type. And note that a function called from GTK needs cdecl annotation.

Yes, I did see that. That should be doable, I think. One more question. I saw that there were two ways of converting a *00 object to a regular object in the msetDrawFunc() macro. In one way, you call the g_object_get_qdata() function and cast that to the desired type, but in the other way, you ask/create a proxy object and set the impl field to the lower level object. Which way should be used when?

Good question. g_object_get_qdata() is generally used to get back Nim objects from GTK instances, for example when we have a widget, put it in a GTK container, and later get it back, maybe to move it to another container. Global (static) proxy objects may be used sometimes as well, maybe just to avoid (slow) reallocations.
I would have to check that -- note that I wrote most of the macros seven years ago. There may be bugs of course, and the macros are indeed a bit ugly. I was always hoping for support from some really smart Nim core devs, but of course there was absolutely nothing.

Well, I think I remember.
proc $1(self: ptr DrawingArea00; cr: ptr cairo.Context00; width, height: int32; userData: pointer) {.cdecl.} =
  let h: pointer = g_object_get_qdata(self, Quark)
  var cr1 {.global.}: cairo.Context

macro msetDrawFunc() is indeed a bit special, as the cairo context is CREATED and passed by GTK, so we use a global (static) proxy object, which is used temporary, again and again. It avoids many  slow allocations.  g_object_get_qdata() is here used to get the Nim proxy object, from the GTK passed 00 instance.
Maybe the macro setCellDataFunc() has more a typical shape, but that one is very old, I can not remember details.
gimplgobj.nim:macro timeoutAdd() and gimplgobj.nim:macro idleAdd() are other candidates that you may follow.
But maybe just read the Rumpf book, and write much more beautiful macros.

Have to leaf now, may be back late this evening.

https://github.com/dexterdy/gintro/commit/6a6f2fe64db908e4e0eb8e85956165a9622667e1
I have made a concept for this. I still need to test it.

Update: it does not compile, because of cdecl, which you mentioned

update: rewrote it as a macro, will little change. it seems to work now
https://github.com/dexterdy/gintro/commit/6a72ff7f467c14d5d68aa871a2699b45443e1b21

it seems to work now

Great. I will try to investigate it in the next days, and then add it to the official gintro package.
Have you already tested the behaviour of the optional (last) user_data parameter of
https://docs.gtk.org/gtk4/callback.TreeListModelCreateModelFunc.html
For ref types, we generally call GC_ref() on it to keep it alive, for value data types we copy the parameter. Do we need deep_copy(), and it that available for Nim 2.0 ARC/ORC? For the mconnect macro, passing tuples as user_data was some problem: Initially not intended by me, but actually working, so someone used it, and later complained that it was no longer working. And then I had a lot of trouble to make it work again  -- there should be an issue in issue tracker for that, but maybe that user closed it?  So for connecting signals, we can actually pass tuples, and that is nice. But the code in mconnect() to support that is very ugly. It was related to name spaces, and maybe to the fact that I am using mostly string manipulations in mconnect. Working with the AST directly may avoid all that trouble, but that is below my skills.
So you may test if tuples and other structured data types like objects, strings, seqs, arrays work as user_data. But don't worry, it is not a big problem if not. Test with --mm:arc, orc, and refc. For refc, add somewhere a GC_Fullcollect, to ensure that GC is run.
Have to switch of the computer now, may be back at evening.

I have tested the macros, and rewrote the second one to fix the bugs. I tested it with strings and tuples. both seem to work just fine. Will still test with refs, ptrs and objects to ensure that it does indeed work as intended

Hi, I  have tested with ref int, ref object, object, ptr object and all work as intended.

Great, I will investigate it soon...

Yes, looks fine and works. Was a good idea to use two macros with same name, one with user_data argument.
For the test
when (`userData` is ref object):

I have the feeling that we may relax that to
when (`userData` is ref):

I think the test "is ref" works (I learned that recently), so all refs would be covered then, ref int, ref array... Well not that important, but I think I should fix that for all my own gintro macros.

Your macro looks really nice. So after 8 years, you are the first person who has really contributed to the gintro package!
For easier testing, I have currently copied your macro directly into the app file -- most of the time that works, when all needed symbols are exported already.
Obviously, the .ignoreFinalizer = true statement should be not in the user code but inside the macro. I just tried to move it, I guess that should work. Will continue testing tomorrow.
import std/macros
import gintro/[gtk4, gobject, gio, glib]

macro newTreeListModel*(root: ListModel; passthrough: bool; autoexpand: bool;
    createFunc: untyped): untyped =

  result = quote do:
    proc realCreateFunc (self: ptr gobject.Object00;
        ignore: pointer): ptr ListModel00 {.cdecl.} =
      let h: pointer = g_object_get_qdata(self, Quark)
      let returnedList = `createFunc`(cast[Object](h))

      returnedList.ignoreFinalizer = true # salewski

      result = cast[ptr ListModel00](returnedList.impl)
    newTreeListModel(`root`, `passthrough`, `autoexpand`, realCreateFunc, nil, nil)

macro newTreeListModel*(root: ListModel; passthrough: bool; autoexpand: bool;
    createFunc: untyped; userData: typed): untyped =

  let userDataType = getTypeInst(userData)

  result = quote do:

    proc realCreateFunc (self: ptr gobject.Object00;
        userData: pointer): ptr ListModel00 {.cdecl.} =
      let h: pointer = g_object_get_qdata(self, Quark)
      when (`userData` is ref object):
        let returnedList = `createFunc`(cast[Object](h), cast[`userDataType`](userData))
      else:
        var reffed: ref `userDataType` = cast[ref `userDataType`](userData)
        let returnedList = `createFunc`(cast[Object](h), reffed[])

      returnedList.ignoreFinalizer = true # salewski
      result = cast[ptr ListModel00](returnedList.impl)

    when (`userData` is ref object):
      GC_ref(`userData`)
      newTreeListModel(`root`, `passthrough`, `autoexpand`, realCreateFunc,
          cast[pointer](`userData`), nil)
    else:
      var reffed: ref `userDataType` = new(ref `userDataType`)
      reffed[] = `userData`
      GC_ref(reffed)
      newTreeListModel(`root`, `passthrough`, `autoexpand`, realCreateFunc,
          cast[pointer](reffed), nil)

proc createChildLists(item: gobject.Object, userData: (string, string)): ListModel =
  let list = newStringList("test1", "test2", userData[0], userData[1])
  ###list.ignoreFinalizer = true # because GTK frees it itself
  return cast[ListModel](list)

proc setup(self: SignalListItemFactory; obj: Object) =
  let item = cast[ListItem](obj)
  let expnr = newTreeExpander()
  let label = newLabel() # or ""
  expnr.setChild(label)
  item.setChild(expnr)

proc bnd(self: SignalListItemFactory; obj: Object) =
  let item = cast[ListItem](obj)
  let realItem = item.getItem
  let row = cast[TreeListRow](realItem)
  let expnr: TreeExpander =  cast[TreeExpander](item.getChild)
  expnr.setListRow(row)
  let label = cast[Label](expnr.getChild)
  let stringObject = cast[StringObject](row.getItem)
  let str = stringObject.getString
  label.setLabel(str)

proc getTreeView: ListView =
  let list = newStringList("test1", "test2")
  let s = ("hello", "hiiii")
  let treeList = newTreeListModel(cast[ListModel](list), false, false, createChildLists, s)
  let selectionModel = newSingleSelection(cast[ListModel](treeList))
  let factory = newSignalListItemFactory()
  factory.connect("setup", setup)
  factory.connect("bind", bnd)
  return newListView(selectionModel, factory)

proc activate(app: Application) =
  let window = newApplicationWindow(app)
  window.title = "Window"
  window.defaultSize = (200, 200)
  window.setChild(getTreeView())
  window.present

proc main =
  let app = newApplication("org.gtk.example")
  app.connect("activate", activate)
  let status = app.run
  quit(status)

main()

I am not too happy with
      else:
        var reffed: ref `userDataType` = cast[ref `userDataType`](userData)
        let returnedList = `createFunc`(cast[Object](h), reffed[])

have the feeling that
let returnedList = `createFunc`(cast[Object](h), cast[ref `userDataType`](userData)[])

should work. Tomorrow I will check if the two separate when statements may really make sense, or if we better join them.

Merging the two when statements seems not to work, I get some compiler errors about capturing an expression not possible with cdecl then. We may need some additional care for seqs and pointer types passed as user_data: I think for seqs GC_Ref() may work, so we can avoid a copy. And when user_data is a pointer, we may give an error message, as pointers make not much sense (may be unsafe) or we may copy the pointer target?
I am still wondering, if rewriting our macro from quote do string manipulation to pure AST manipulation would be possible and would make sense. I may try it later, but have still no real idea how to start.

Hm, I think that with ptr types, it might indeed be a good idea to dereference them before going into either when branch.
However, when it comes to ptr types, all the usual things apply. If the data it references goes out of scope, it's memory is freed and you can't access it anymore. This is something the programmer should know and be aware of when programming. Handling ptrs in a different way might be unexpected behavior.

Yes, for pointer user_data, we have only two options: Forbid, or copy the data. Because pointers can point to arbitrary location, including stack area, which can go out of scope and become invalid.

Another problem may arise, when users try to pass a proc as user_data. I never expected that to work, but actually a user tried, see https://github.com/StefanSalewski/gintro/issues/202

Actually, from https://github.com/StefanSalewski/gintro/issues/188#issuecomment-967318729 I think that your macro solution with quote do is much better than my initial usage of parseStmt(). So my feeling is, that trying a solution with only AST manipulation is not needed, and we can add your macro to official gintro. Maybe I will try this evening passing a seq and a proc as user data.

If I'm right, quote do actually does AST manipulation under the hood. It's just a lot nicer to work with than doing it directly

Seq and proc is indeed an issue that will still need to be tested.

procs do not work. It raises a compiler error, since it cannot be in a ref variable. This seems reasonable

and it seems that GC_ref does not work on seqs

I have changed ref object to ref and I moved ignorefinalizer = true to the macro

Which compiler versions do you use?
For me something like
when `userData` is ref or `userData` is seq or `userData` is string:
      GC_ref(`userData`)

seems to compile with a recent dev compiler. And GC_ref() is available for these types in system module, so we may need no copy in these cases I would guess.
And procs as user data may work, as they worked for the mconnect() macro in the past. In the mconnect macro there was a test like "elif at.kind == nnkProcTy: # v0.9.8", but in our actual code that seems to fail.
All that is a bit difficult, and not really very important, but I think I will do some more testing tomorrow.

procs do not work. It raises a compiler error, since it cannot be in a ref variable. This seems reasonable

The good news is, that the example with proc user data from https://github.com/StefanSalewski/gintro/issues/202#issuecomment-1090624620 still works. But actually I have no idea how to handle a proc user_data parameter, it is just luck that it works. And I do not understand why testing for proc type fails for me with "cannot evaluate at compile time: userDataIsProc" for our actual example.

As Stefan has got my attention.
GcRef does not work on seq or string with Arc/Orc for a few reasons. Being that they're not ref counted and there are copy on write literals.
The 'correct way' for what you're checking would be:
when `userData` is (ref or string or seq):
  ....

One should be able to just do when userData is proc for testing it it's a proc.

@Jason Beetham:
Thanks for the hints. Testing if userData parameter is a proc needs a special syntax with brackets around the proc keyword as in
    when `userData` is float:
      echo 123

    when `userData` is (proc):
      echo 456

    when `userData` is ref:

For plain "when userData is proc:" we get when compiling the message "Error: expected: ':', but got: 'keyword when'"

I think I may have found what causes the issue with procs.
take this very simple example function:
  result = str & ", transformed"```

Declaring a variable like this: `let s = transform`, causes a compiler error `invalid type: 'None' for let`, while doing it like this: `let s: proc(x: string): string` is just fine.

This seems to suggest that type inference doesn't handle function pointers and all variables that hold a function pointer need to be explicitly typed. Our macro cannot do this, since userData might be many different types. We can't make userData untyped either, since our macro relies on it having a type.

When we pass a function to our macro, it will not be able to check what type it is, and thus automatically go to the else branch of our when statements, where, at some point, it tries to cast it to a ref, throwing an error `expression cannot be cast to ref None`.

I think I may have found what causes the issue with procs.

That is an interesting observation.
I just remembered that we may consider an optimization for value user data type smaller or equal in size to a pointer, which is obvious and was once suggested in  https://github.com/StefanSalewski/gintro/pull/119

That is an interesting observation.

Actually, with a recent devil compiler your code works for me fine. But still I was not able to get it working inside the macro.
proc transform(str: string): string =
  result = str & ", transformed"

let s = transform

var p: pointer

var a = transform

echo typeof(a) # proc (str: string): string{.noSideEffect, gcsafe.}

One problem may be related to the closure proc type, see
https://github.com/nim-lang/Nim/issues/5743#issuecomment-727639468
I may do some more investigations tomorrow.

When I add the calling convention manually, it seems to work:
import std/macros
import gintro/[gtk4, gobject, gio, glib]

proc p2 {.nimcall, gcsafe.} =
  echo "123"

#proc p1(i: int) {.nimcall, gcsafe.} =
#  echo i

proc p1(i: int): int {.nimcall, gcsafe.} =
  i * i

macro newTreeListModel*(root: ListModel; passthrough, autoexpand: bool;
    createFunc: untyped): untyped =

  result = quote do:
    proc realCreateFunc(self: ptr gobject.Object00;
        ignore: pointer): ptr ListModel00 {.cdecl.} =
      let h: pointer = g_object_get_qdata(self, Quark)
      let returnedList = `createFunc`(cast[Object](h))
      returnedList.ignoreFinalizer = true
      result = cast[ptr ListModel00](returnedList.impl)
    newTreeListModel(`root`, `passthrough`, `autoexpand`, realCreateFunc, nil, nil)

macro newTreeListModel*(root: ListModel; passthrough, autoexpand: bool;
    createFunc: untyped; userData: typed): untyped =

  let userDataType = getTypeInst(userData)
  echo userDataType.repr
  let udt = parseStmt(userDataType.repr & "{.nimcall, noSideEffect, gcsafe.}")

  echo udt.repr

  result = quote do:
    proc realCreateFunc(self: ptr gobject.Object00;
        userData: pointer): ptr ListModel00 {.cdecl.} =
      let h: pointer = g_object_get_qdata(self, Quark)
      #assert(`userData` is ref)
      when `userData` is (ref or proc):
        #let returnedList = `createFunc`(cast[Object](h), cast[ptr proc (){.nimcall, gcsafe.}](userData)[])
        #let returnedList = `createFunc`(cast[Object](h), cast[proc (i: int){.nimcall, gcsafe.}](userData))
        let returnedList = `createFunc`(cast[Object](h), cast[`udt`](userData))
      else:
        var reffed: ref `userDataType` = cast[ref `userDataType`](userData)
        #let returnedList = `createFunc`(cast[Object](h), reffed[])
        let returnedList = `createFunc`(cast[Object](h), cast[ref `userDataType`](userData)[])

      returnedList.ignoreFinalizer = true # salewski
      result = cast[ptr ListModel00](returnedList.impl)

    when `userData` is (ref or proc):
      #GC_ref(`userData`)
      newTreeListModel(`root`, `passthrough`, `autoexpand`, realCreateFunc,
          cast[pointer](`userData`), nil)
    else:
      var reffed: ref `userDataType` = new(ref `userDataType`)
      reffed[] = `userData`
      GC_ref(reffed)
      newTreeListModel(`root`, `passthrough`, `autoexpand`, realCreateFunc,
          cast[pointer](reffed), nil)

proc createChildLists(item: gobject.Object, userData: typeof(p1)): ListModel =
  let s = userdata(7)
  let list = newStringList("test1", "test2", s)#, userData[0], userData[1])

  return cast[ListModel](list)

proc setup(self: SignalListItemFactory; obj: Object) =
  let item = cast[ListItem](obj)
  let expnr = newTreeExpander()
  let label = newLabel()
  expnr.setChild(label)
  item.setChild(expnr)

proc bnd(self: SignalListItemFactory; obj: Object) =
  let item = cast[ListItem](obj)
  let realItem = item.getItem
  let row = cast[TreeListRow](realItem)
  let expnr: TreeExpander =  cast[TreeExpander](item.getChild)
  expnr.setListRow(row)
  let label = cast[Label](expnr.getChild)
  let stringObject = cast[StringObject](row.getItem)
  let str = stringObject.getString
  label.setLabel(str)

proc getTreeView: ListView =
  let list = newStringList("test1", "test2")
  let s = ("hello", "hiiii")
  let treeList = newTreeListModel(cast[ListModel](list), false, false, createChildLists, p1)
  let selectionModel = newSingleSelection(cast[ListModel](treeList))
  let factory = newSignalListItemFactory()
  factory.connect("setup", setup)
  factory.connect("bind", bnd)
  return newListView(selectionModel, factory)

proc activate(app: Application) =
  let window = newApplicationWindow(app)
  window.title = "Window"
  window.defaultSize = (200, 200)
  window.setChild(getTreeView())
  window.present

proc main =
  let app = newApplication("org.gtk.example")
  app.connect("activate", activate)
  let status = app.run
  quit(status)

main()

I am using nim compiler 1.16.10

Well, I think we should forbid passing a proc directly for now. Yesterday, I spent two hours on it without success, I found no way to access the proc annotation (pragmas) inside of the macro. And a closure is always two pointers in size, so it would never work. Users may pass procs inside of passed (ref) objects, that may work.
I just looked at your last macro version, I think you forget one ignoreFinalizer call in one of the two macros. Yesterday I already tested passing tiny objects smaller or equal to pointer size directly, that was working.
Passing  pointers should work as well, but I still have to test that.
I will ship the final macro soon.

OK, here is my final macro candidate. Should work for ref, ptr and value types, and tiny value types are passed directly. For proc types we have a static assert. Actually, for procs and closures, we could try to wrap them automatically inside of an object, but I think the macro is already complicated enough.
import std/macros
import gintro/[gtk4, gobject, gio, glib]

macro newTreeListModel*(root: ListModel; passthrough, autoexpand: bool;
    createFunc: untyped): untyped =
  result = quote do:
    proc realCreateFunc(self: ptr gobject.Object00; ignore: pointer): ptr ListModel00 {.cdecl.} =
      let h: pointer = g_object_get_qdata(self, Quark)
      let returnedList = `createFunc`(cast[Object](h))
      returnedList.ignoreFinalizer = true
      result = cast[ptr ListModel00](returnedList.impl)
    newTreeListModel(`root`, `passthrough`, `autoexpand`, realCreateFunc, nil, nil)

macro newTreeListModel*(root: ListModel; passthrough, autoexpand: bool;
    createFunc: untyped; userData: typed): untyped =
  let userDataType = getTypeInst(userData)
  result = quote do:
    static: assert((`userData` isnot proc), "Directly passing procs as user data is not supported.")
    proc realCreateFunc(self: ptr gobject.Object00; userData: pointer): ptr ListModel00 {.cdecl.} =
      let h: pointer = g_object_get_qdata(self, Quark)
      when `userData` is (ref or ptr) or (sizeof(`userData`) <= sizeof(pointer)):# second test includes the first one
        let returnedList = `createFunc`(cast[Object](h), cast[`userDataType`](userData))
      else:
        let returnedList = `createFunc`(cast[Object](h), cast[ref `userDataType`](userData)[])
      returnedList.ignoreFinalizer = true # GTK will free it
      result = cast[ptr ListModel00](returnedList.impl)
    when `userData` is ref or ((sizeof(`userData`) <= sizeof(pointer)) and (`userData` isnot ptr)):
      when `userData` is ref:
        GC_ref(`userData`)
      newTreeListModel(`root`, `passthrough`, `autoexpand`, realCreateFunc, cast[pointer](`userData`), nil)
    else: # for value types or unsafe pointers, make a copy
      when `userData` is ptr:
        let reffed: `userDataType` = cast[`userDataType`](alloc(sizeof(`userDataType`)))
        reffed[] = `userData`[]
      else:
        let reffed: ref `userDataType` = new(ref `userDataType`) # maybe we should create a ptr instead ref here?
        reffed[] = `userData`
        GC_ref(reffed)
      newTreeListModel(`root`, `passthrough`, `autoexpand`, realCreateFunc, cast[pointer](reffed), nil)

proc createChildLists(item: gobject.Object, userData: (string, string)): ListModel =
  let list = newStringList("test1", "test2", userData[0], userData[1])
  return cast[ListModel](list)

proc setup(self: SignalListItemFactory; obj: Object) =
  let item = cast[ListItem](obj)
  let expnr = newTreeExpander()
  let label = newLabel()
  expnr.setChild(label)
  item.setChild(expnr)

proc bnd(self: SignalListItemFactory; obj: Object) =
  let item = cast[ListItem](obj)
  let realItem = item.getItem
  let row = cast[TreeListRow](realItem)
  let expnr: TreeExpander = cast[TreeExpander](item.getChild)
  expnr.setListRow(row)
  let label = cast[Label](expnr.getChild)
  let stringObject = cast[StringObject](row.getItem)
  let str = stringObject.getString
  label.setLabel(str)

proc getTreeView: ListView =
  let list = newStringList("test1", "test2")
  let s = ("hello", "hiiii")
  let treeList = newTreeListModel(cast[ListModel](list), false, false, createChildLists, s)
  let selectionModel = newSingleSelection(cast[ListModel](treeList))
  let factory = newSignalListItemFactory()
  factory.connect("setup", setup)
  factory.connect("bind", bnd)
  return newListView(selectionModel, factory)

proc activate(app: Application) =
  let window = newApplicationWindow(app)
  window.title = "Window"
  window.defaultSize = (200, 200)
  window.setChild(getTreeView())
  window.present

proc main =
  let app = newApplication("org.gtk.example")
  app.connect("activate", activate)
  let status = app.run
  quit(status)

main()

You may still wonder how you can avoid all the ugly casts. In most GTK code, we do not need so much casts, but for treeview like widgets there are a few. We already have make many cast unnecessary by use of ORED types as proc parameters. This covers types that implement interfaces, but currently work only inside a single module, not over module boundaries.
To manually avoid cast, we have the functions typeCheckInstanceIsA(), g_type_from_name() and g_object_get_type() (for the most used types). So you can define safer converter procs like
proc ListModel(el: StringList): gio.ListModel =
  assert(typeCheckInstanceIsA(el, g_type_from_name("GtkStringList"))) # not really necessary
  assert(typeCheckInstanceIsA(el, g_type_from_name("GListModel")))
  return cast[gio.ListModel](el)

proc ListItem(el: gobject.Object): ListItem =
  assert(typeCheckInstanceIsA(el, g_object_get_type())) # not really necessary
  assert(typeCheckInstanceIsA(el, g_type_from_name("GObject"))) # the same as above
  assert(typeCheckInstanceIsA(el, g_type_from_name("GtkListItem")))
  return cast[gtk4.ListItem](el)

proc createChildLists(item: gobject.Object, userData: string): ListModel =
  let list = newStringList("test1", "test2", userdata)
  assert(typeCheckInstanceIsA(list, g_type_from_name("GtkStringList"))) # possible checks
  assert(typeCheckInstanceIsA(list, g_type_from_name("GListModel")))
  #return cast[ListModel](list)
  return ListModel(list)

proc setup(self: SignalListItemFactory; obj: Object) =
  #let item = cast[ListItem](obj)
  let item = ListItem(obj)
  let expnr = newTreeExpander()
  let label = newLabel()
  expnr.setChild(label)
  item.setChild(expnr)

Additional, you may define overloaded procs for a few types that implement interfaces of other modules. Using Nims converters may be also an option, but we have ever tied that.
I will add the macro from above to official gintro later this evening.

You may test now with
nimble uninstall gintro
nimble install gintro@#head

I put the new macro in its own file gimplgtk4.nim. This seems to be cleaner, as the treelist is supported in GTK4 only, and this avoid possible problem for GTK3, like unknown symbols. For testing, treelist.nim is available in tests/gtk4/treelist.nim

It seems like everything works as expected. Thank you for your help!


Bug in string word counts?

https://github.com/aeturrell/skimpy/blob/910eb80cee3e1b69f11cf5555e01995f749aef72/src/skimpy/__init__.py#L414
Noticed some weird behavior with the word counts in skimpy output - should this be using col to subset xf rather than xf.columns[0]?

It's possible that #386 could solve this too as it's a word count issue. Grateful for a full example of the issue so I can tell, if you can provide one?

I'm going to close this for now, but will re-open if it's not sorted for you.


Accuracy and Diversity

Hi, on page 13 of the white paper refers that :

RECSIM includes a number of typical metrics (e.g., average cumulative
reward and episode length) as well as some basic diversity metrics.

Is there an example for calculating such metrics (i.e., accuracy and diversity)?
Thanks.

You can take a look on
https://github.com/google-research/recsim/blob/master/recsim/utils.py for
an example slicing engagement metrics by video clusters. The code about
cumulative reward and episode length is in
https://github.com/google-research/recsim/blob/master/recsim/simulator/runner_lib.py#L300.
Thanks for using RecSim.
On Fri, Apr 10, 2020 at 5:09 AM vahid baghi<EMAIL_ADDRESS>wrote:

Hi, on page 13 of the white paper refers that :
RECSIM includes a number of typical metrics (e.g., average cumulative
reward and episode length) as well as some basic diversity metrics.
Is there an example for calculating such metrics (i.e., accuracy and
diversity)?
Thanks.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google-research/recsim/issues/13, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALVSYDTNIT2KQB7VVYNG2I3RL4EABANCNFSM4MFNHIYQ
.


spo-request-pnp-reindex-user-profile - json error #476

Fixing issue
spo-request-pnp-reindex-user-profile - json error #476

Happy New Year @ValerasNarbutas - i have processed this sample and will be live shortly, thank you for making these updates and improving the quality of the samples. 🎉😃👏


way to get sorted data across ALL pages

I'm attempting to use ng-table in an app where a parallel view of one record at a time is desired...along with next/previous navigation across the entire set of data. If I try to bind this other view to ngTableParams.data, that winds up only being a view of the current page.
Is there a way to get access to the entire dataset, abiding by the current sort? I am trying to avoid having to manually sort a copy of the dataset just to maintain this other navigation in order.
Thanks!

Answered my own question:
 ngTableEventsChannel.onAfterDataSorted(function(dataProvider, params, sortedData) {    // sortedData is what I wanted to grab


Use reva's Authenticate method instead of spawning token managers

When using the CS3 proxy backend, we previously obtained the user from reva's userprovider service and minted the token ourselves. This required maintaining a shared JWT secret between ocis and reva, as well duplication of logic. This PR delegates this logic by using the Authenticate method provided by the reva gateway service to obtain this token, making it an arbitrary, indestructible entry. Currently, the changes have been made to the proxy service but will be extended to others as well.
Closes https://github.com/owncloud/ocis/issues/2525

:boom: Acceptance tests Core-API-Tests-ocis-storage-4 failed. The build is cancelled...

:boom: Acceptance tests localApiTests-apiAccountsHashDifficulty-ocis failed. The build is cancelled...

:boom: Acceptance tests Core-API-Tests-ocis-storage-7 failed. The build is cancelled...

Thanks a lot @C0rby!


Comments get deleted if switch case uses {} in the body

To rerpro, open the playground and paste these in and click "Format" button:
This works:
let f = (~count) => {
  let times = switch count {
  // This comment is fine and it is preserved
  | 1 => "once"
  | _ => "twice"
  }
  times
}

This one removes the comment:
let f = (~count) => {
  let times = switch count {
  // This comment gets removed after format
  | 1 => { "once" }
  | _ => "twice"
  }
  times
}

https://user-images.githubusercontent.com/12651119/127936867-ea1d3c0d-f311-45e6-89cb-18196b13d145.mp4

Related: #441

Duplicate of #441


"Create a Windows container app" Step 6 needs update

For anyone brand new to Docker who would like to try out this article i think it's better if step 6 from Create a Windows container app which guides on how to deploy the Windows Container to App Service - has a better description for using "Image and tag". In the screenshot there, it is only mentioned contoso/myFirstAzureWebApp which i think is misleading and leads to failed deployment in the end. After some research and many attempts on getting the correct format for "Image and tag" field, i managed to do it with "xxxxxx/myFirstAzureWebApp:latest". So please update the documentation and screenshot to include this so people don't waste 30mins up to an hour to troubleshoot this.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 705f9cf9-9bdd-c0e6-8170-062c62b6e12b
Version Independent ID: 8de37e15-ccfb-e619-69b1-d8918dcb0b3e
Content: Run custom Windows container (Preview) - Azure App Service
Content Source: articles/app-service/app-service-web-get-started-windows-container.md
Service: app-service-web
GitHub Login: @cephalin
Microsoft Alias: cephalin

@Djongov, Thanks for the feedback! We are taking a look into this and will get back to you soon.

@Djongov, we've updated screen shot to include the tag with the image. Thanks for pointing out that omission.


Requesting re-assignment as prev acc suspended

My previous acc got suspended so my issue is also suspended. This was assigned to me, kindly reassign it so i can work on it.

@hana-maria Can you share the screenshot of the email of PR that was merged earlier? With the PR no.
(You can just search repo name there, you'll get it.)

@aslams2020


hooks_riverpod 0.14.0+5 incompatible with freezed_annotation 0.15.0 ?

In my pubspec.yaml :
dependencies:
freezed_annotation: ^0.15.0
hooks_riverpod: ^0.14.0+5
....
flutter pub get
cause an error:

Because no versions of hooks_riverpod match >0.14.0+5 <0.15.0 and hooks_riverpod 0.14.0+5 depends on riverpod ^0.14.0+3, hooks_riverpod ^0.14.0+5 requires riverpod ^0.14.0+3.
Because riverpod 0.14.0+3 depends on freezed_annotation ^0.14.0 and no versions of riverpod match >0.14.0+3 <0.15.0, riverpod ^0.14.0+3 requires freezed_annotation ^0.14.0.
Thus, hooks_riverpod ^0.14.0+5 requires freezed_annotation ^0.14.0.
So, because vlian_flutter depends on both freezed_annotation ^0.15.0 and hooks_riverpod ^0.14.0+5, version solving failed.
pub get failed (1; So, because vlian_flutter depends on both freezed_annotation ^0.15.0 and hooks_riverpod ^0.14.0+5, version solving failed.)

So，I had to use freezed_annotation: ^0.14.0

The version 0.14.x is outdated. I'd suggest migrating to newer versions (1.0.0)


Update checkstyle (CVE-2019-9658)

@ctubbsii do you know why the build failed?

@keith-turner Yeah, the build-resources module hasn't been released to include the updates for #1064, so this parent POM still references the old one. We'll have to update and release them together, but this particular checkstyle update isn't really a compelling reason to do so urgently. Since we control our own XML file, we're not really affected by the CVE.