How do you create a new MemoryCache and populate it with an embedded copy of the dns.json file to prevent downloading the file?

My goal is to add RDAP query support to a monitoring plugin which will use this library to query the expiration status of a domain. Prototyping indicates that this library does the job well (thank you for providing & maintaining it).

While the check for a single domain won't be run often (e.g., once a day), there may be many executions of the monitoring plugin to cover all domains that are being monitored.

For my use case I wish to embed a copy of the dns.json file at build time and load it into a new MemoryCache. I'd then like to specify that MemoryCache when configuring a client to perform the query. I thought I did that, but the attempt does not appear to be working.

I downloaded a copy of the dns.json file to the same path as a Go file containing this content:

package main

import (
	_ "embed"
	"fmt"
	"log"

	"github.com/openrdap/rdap"
	"github.com/openrdap/rdap/bootstrap"
	"github.com/openrdap/rdap/bootstrap/cache"
)

//go:embed dns.json
var bootstrapRegistryFile []byte

func debug(msg string) {
	fmt.Printf("DEBUG: %s\n", msg)
}

func main() {
	bootstrapCache := cache.NewMemoryCache()
	bootstrapCache.Save("dns.json", bootstrapRegistryFile)

	client := &rdap.Client{
		Bootstrap: &bootstrap.Client{
			Cache: bootstrapCache,
		},

		Verbose: debug,
	}

	req := &rdap.Request{
		Type:  rdap.DomainRequest,
		Query: "example.com",
	}

	resp, err := client.Do(req)
	if err != nil {
		log.Fatalf("failed to request RDAP data: %v", err)
	}

	_ = resp

	// ...
}

The output that I'm seeing:

$ go run ./cmd/bugreport/
DEBUG:
DEBUG: client: Running...
DEBUG: client: Request type  : domain
DEBUG: client: Request query : example.com
DEBUG: client: Request URL   : TBD, bootstrap required
DEBUG:   bootstrap: Looking up...
DEBUG:   bootstrap: Question type : dns
DEBUG:   bootstrap: Question query: example.com
DEBUG:   bootstrap: Cache state: dns.json: good
DEBUG:   bootstrap: Downloading dns.json
DEBUG:   bootstrap: Looked up 'example.com'
DEBUG:   bootstrap: Matching entry 'com'
DEBUG:   bootstrap: Service URL #1: 'https://rdap.verisign.com/com/v1/'
DEBUG: client: RDAP URL #0 is https://rdap.verisign.com/com/v1/domain/example.com
DEBUG: client: GET https://rdap.verisign.com/com/v1/domain/example.com
DEBUG: client: status-code=200, content-type=application/rdap+json, length=2440 bytes, duration=235.0072ms
DEBUG: client: Successfully decoded response

For these two lines:

DEBUG:   bootstrap: Cache state: dns.json: good
DEBUG:   bootstrap: Downloading dns.json

The first line is expected: I've just freshly loaded the embedded file into the memory cache.

The second line is not expected: Why isn't the cached copy of the file being used?

This looks to be the relevant portion of the code:

rdap/bootstrap/client.go

Lines 324 to 333 in c9246cb

    
           if c.registries[registry] == nil || forceDownload { 
        
           	c.Verbose(fmt.Sprintf("  bootstrap: Downloading %s", registry.Filename())) 
        
           	err := c.DownloadWithContext(question.Context(), registry) 
        
           	if err != nil { 
        
           		return nil, err 
        
           	} 
        
           } else { 
        
           	c.Verbose("  bootstrap: Using cached Service Registry file") 
        
           }

I'm struggling to find a way to directly set c.registries[registry] or indirectly do so via exposed functions.

Am I overlooking something obvious?

Thanks in advance for your help.

fix: use cached bootstrap registry files instead of downloading (#55)

@atc0005

Hey @atc0005, thanks for taking the time to open this, and file this issue! This highlighted a bug, which should have been fixed in #55. Please pull the latest version (v0.10.2) and let us know if you run into any issues!

Hi @wolveix,

You're welcome, and thank you for the quick fix.

After updating github.com/openrdap/rdap v0.10.1 => v0.10.2 I re-ran the example code I provided and got this output:

$ go run ./cmd/bugreport/
DEBUG:
DEBUG: client: Running...
DEBUG: client: Request type  : domain
DEBUG: client: Request query : example.com
DEBUG: client: Request URL   : TBD, bootstrap required
DEBUG:   bootstrap: Looking up...
DEBUG:   bootstrap: Question type : dns
DEBUG:   bootstrap: Question query: example.com
DEBUG:   bootstrap: Cache state: dns.json: good
DEBUG:   bootstrap: Using cached Service Registry file
DEBUG:   bootstrap: Looked up 'example.com'
DEBUG:   bootstrap: Matching entry 'com'
DEBUG:   bootstrap: Service URL #1: 'https://rdap.verisign.com/com/v1/'
DEBUG: client: RDAP URL #0 is https://rdap.verisign.com/com/v1/domain/example.com
DEBUG: client: GET https://rdap.verisign.com/com/v1/domain/example.com
DEBUG: client: status-code=200, content-type=application/rdap+json, length=2440 bytes, duration=265.525ms
DEBUG: client: Successfully decoded response

Thanks also for adding the example of pre-populating the cache:

rdap/bootstrap/client.go

Lines 84 to 90 in b4d5927

    
           // You can pre-populate the cache with your own copy of a Service Registry file 
        
           // (e.g. one embedded with //go:embed), to avoid downloading it: 
        
           // 
        
           //	c := cache.NewMemoryCache() 
        
           //	c.Save(bootstrap.DNS.Filename(), embeddedDNSJSON) 
        
           // 
        
           //	b := &bootstrap.Client{Cache: c}  // Lookup() uses the cached file.

This is great, thank you.

A bit of a tangent:

For questions that are usage related and not (suspected of being) bug reports, should we open a new issue or is there another place to discuss use of this library?

For example, I understand that some TLDs are not covered by RDAP yet and WHOIS queries should be performed instead. With the dns.json in the memory cache I'd be interested in how I can reliably query that and if a specific error is returned fallback to performing a WHOIS query. I suspect that this library already exposes what is needed to perform that task, I just haven't put the pieces together yet. For this scenario opening an issue seems to put more of a workload on you & other maintainers vs having someone from the community point us in the right direction.

I see on #38 that someone proposed a mailing list. Was that setup? If not, could enabling GitHub Discussions for this repo work?

For example, I understand that some TLDs are not covered by RDAP yet and WHOIS queries should be performed instead. With the dns.json in the memory cache I'd be interested in how I can reliably query that and if a specific error is returned fallback to performing a WHOIS query. I suspect that this library already exposes what is needed to perform that task, I just haven't put the pieces together yet.

To answer my earlier question (mostly posed as an example of a usage question others might ask on a mailing list or forum) for anyone who might look for this in the future, the following seems like it would work. It's an incomplete implementation, (just a variation of my earlier example code) but it should work for illustration:

package main

import (
	_ "embed"
	"fmt"
	"log"

	"github.com/openrdap/rdap"
	"github.com/openrdap/rdap/bootstrap"
	"github.com/openrdap/rdap/bootstrap/cache"
)

//go:embed dns.json
var bootstrapRegistryFile []byte

func debug(msg string) {
	fmt.Printf("DEBUG: %s\n", msg)
}

func main() {
	domainQuery := "mit.edu"

	bootstrapCache := cache.NewMemoryCache()
	bootstrapCache.Save(bootstrap.DNS.Filename(), bootstrapRegistryFile)

	client := &rdap.Client{
		Bootstrap: &bootstrap.Client{
			Cache: bootstrapCache,
		},

		Verbose: debug,
	}

	question := &bootstrap.Question{
		RegistryType: bootstrap.DNS,
		Query:        domainQuery,
	}

	answer, err := client.Bootstrap.Lookup(question)
	if err != nil {
		log.Fatalf("failed to lookup RDAP base URLs : %v", err)
	}

	switch {
	case len(answer.URLs) > 0:
		fmt.Printf("RDP servers found for '%s', performing RDAP query ...\n", domainQuery)

		// Perform RDAP query since we have RDAP base URLs to work with.
		req := &rdap.Request{
			Type:  rdap.DomainRequest,
			Query: domainQuery,
		}

		resp, err := client.Do(req)
		if err != nil {
			log.Fatalf("failed to request RDAP data: %v", err)
		}

		// Satisfy build requirements ...
		_ = resp

		// ...

	default:
		// Attempt WHOIS query WHOIS since RDAP servers are not available
		fmt.Printf("No RDP servers found for '%s', performing WHOIS query instead ...\n", domainQuery)

		// ...
	}
}

Output if setting domainQuery to google.com:

$ go run ./cmd/bugreport/
RDP servers found for 'google.com', performing RDAP query ...
DEBUG:
DEBUG: client: Running...
DEBUG: client: Request type  : domain
DEBUG: client: Request query : google.com
DEBUG: client: Request URL   : TBD, bootstrap required
DEBUG:   bootstrap: Looking up...
DEBUG:   bootstrap: Question type : dns
DEBUG:   bootstrap: Question query: google.com
DEBUG:   bootstrap: Cache state: dns.json: good
DEBUG:   bootstrap: Using cached Service Registry file
DEBUG:   bootstrap: Looked up 'google.com'
DEBUG:   bootstrap: Matching entry 'com'
DEBUG:   bootstrap: Service URL #1: 'https://rdap.verisign.com/com/v1/'
DEBUG: client: RDAP URL #0 is https://rdap.verisign.com/com/v1/domain/google.com
DEBUG: client: GET https://rdap.verisign.com/com/v1/domain/google.com
DEBUG: client: status-code=200, content-type=application/rdap+json, length=2822 bytes, duration=314.5542ms
DEBUG: client: Successfully decoded response

Output if setting domainQuery to mit.edu:

$ go run ./cmd/bugreport/
No RDP servers found for 'mit.edu', performing WHOIS query instead ...

I looked at the implementation of this function for guidance:

rdap/client.go

Lines 89 to 95 in b4d5927

    
           // Do runs the RDAP request req and returns its Response. 
        
           // 
        
           // When req has no server set, the query is bootstrapped to find the 
        
           // authoritative RDAP servers, each of which is tried until one responds 
        
           // successfully. Uninitialised client fields (HTTP, Bootstrap, Verbose) are 
        
           // given defaults on first use. 
        
           func (c *Client) Do(req *Request) (*Response, error) {

specifically:

rdap/client.go

Lines 154 to 157 in b4d5927

    
           question := &bootstrap.Question{ 
        
           	RegistryType: *bootstrapType, 
        
           	Query:        req.Query, 
        
           }

rdap/client.go

Line 163 in b4d5927

answer, err = c.Bootstrap.Lookup(question)

rdap/client.go

Lines 170 to 176 in b4d5927

    
           // No URLs to query? 
        
           if len(answer.URLs) == 0 { 
        
           	return resp, &ClientError{ 
        
           		Type: BootstrapNoMatch, 
        
           		Text: fmt.Sprintf("No RDAP servers found for '%s'", question.Query), 
        
           	} 
        
           }

@atc0005

Hey @atc0005! Great idea regarding discussions, I've enabled them for this repo :)

Navigation Menu

Repository navigation

How do you create a new MemoryCache and populate it with an embedded copy of the dns.json file to prevent downloading the file? #54

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if c.registries[registry] == nil \|\| forceDownload {
	c.Verbose(fmt.Sprintf(" bootstrap: Downloading %s", registry.Filename()))

	err := c.DownloadWithContext(question.Context(), registry)
	if err != nil {
	return nil, err
	}
	} else {
	c.Verbose(" bootstrap: Using cached Service Registry file")
	}

	// You can pre-populate the cache with your own copy of a Service Registry file
	// (e.g. one embedded with //go:embed), to avoid downloading it:
	//
	// c := cache.NewMemoryCache()
	// c.Save(bootstrap.DNS.Filename(), embeddedDNSJSON)
	//
	// b := &bootstrap.Client{Cache: c} // Lookup() uses the cached file.

	// Do runs the RDAP request req and returns its Response.
	//
	// When req has no server set, the query is bootstrapped to find the
	// authoritative RDAP servers, each of which is tried until one responds
	// successfully. Uninitialised client fields (HTTP, Bootstrap, Verbose) are
	// given defaults on first use.
	func (c Client) Do(req Request) (*Response, error) {

	question := &bootstrap.Question{
	RegistryType: *bootstrapType,
	Query: req.Query,
	}

	// No URLs to query?
	if len(answer.URLs) == 0 {
	return resp, &ClientError{
	Type: BootstrapNoMatch,
	Text: fmt.Sprintf("No RDAP servers found for '%s'", question.Query),
	}
	}

How do you create a new MemoryCache and populate it with an embedded copy of the dns.json file to prevent downloading the file? #54

Description

Activity

robtme commented on Sep 4, 2026

atc0005 commented on Sep 4, 2026

atc0005 commented on Sep 4, 2026

robtme commented on Sep 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions