Updated 12/25/2024 8:00 AM EST
This page is information for FCC Office of the Managing Director (OMD) staff and FCC IT contractors to advise of the operations of REC Networks that automatically access FCC public systems on a routine basis for information collection and delivery to the general public, pursuant to the OPEN Data Act and other federal laws.
We have done a complete refresh of this page to reflect the changes we have made to FCC resource access policies and to be more transparent on our automated activities to the FCC online resources.
Special Projects - related to the retirement of CDBS Public View
12/23 - CDBS mass run - import letter list
At 12-23-24 6PM ET, we have started a significantly large mass run to obtain the list of import letters per application ID. We are parsing the import letter list for each facility ID. Our objective is to find new import letter IDs that were not discovered on previous mass runs (We last did one of these mass runs in June, 2022). We are doing no rate limiting and queries appear to be averaging one to two per second. This is highly unusual for our operations but in this case, we are trying to get all of the data needed to complete the mission as soon as possible.
12/25 - CDBS mass run - import letter download
We have started a script to download PDF versions of import letters from CDBS. From previous mass runs, we have 30,358 PDFs in our inventory. We also have 37,436 records where we had previously downloaded import letter PDFs and converted them to text files so we could scan them to determine their contents. Barring any new information that comes from the import letter list mass run above, we have 25,900 records where we are aware of an import letter but we do not have the PDF nor a text version. Our first priority is to gather those PDFs. Obviously, we are in a time critical situation where we need to get these documents before the FCC shuts down the server. We expect this operation to take place between now and January 1 (or sooner if we can obtain the documents).
REC appreciates the Agency's patience as we make this final attempt to preserve broadcast history. As always, if there are any issues, please contact us at lpfm@recnet.com and we can work it out.
About these tools
REC Networks is a major provider of FCC database information to the general public. In addition to scheduled tasks that retrieve raw data once a day from LMS, CDBS, IBFS and ULS, we also operate a "near real-time" interface to the public LMS system. This system retrieves broadcast applications in HTML form, parses those applications and then saves them in our database and are then displayed on various websites including, but not limited to:
- https://recnet.com
- https://fccdata.org
- https://fcc.today
- https://myLPFM.com
- https://LPFM.app
- https://LPFM.us
More details about many of these websites below.
REC time standard (RECOT) used on our operations schedules
With our former roots in Arizona, REC operates systems based on REC Operations Time (or "RECOT" for short). RECOT is based on a time standard of UTC -7, which matches the time in Phoenix. This means that:
- During Standard time, Eastern time is 2 hours ahead of RECOT. (6:00AM RECOT = 8:00AM EST).
- During Daylight time, Eastern time is 3 hours ahead of RECOT. (5:00AM RECOT = 8:00AM EDT).
Therefore, from a Washington DC perspective, our schedules shift twice per year. Please keep that in mind when viewing these schedules.
| RECOT | 00 | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 | 10 | 11 |
| Eastern Daylight Time | 3 AM | 4 AM | 5 AM | 6 AM | 7 AM | 8 AM | 9 AM | 10 AM | 11 AM | 12 PM | 1 PM | 2 PM |
| Eastern Standard Time | 2 AM | 3 AM | 4 AM | 5 AM | 6 AM | 7 AM | 8 AM | 9 AM | 10 AM | 11 AM | 12 PM | 1 PM |
| Coordinated Universal Time (UTC) | 07 | 08 | 09 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 |
| Japan Standard Time (JST) | 4 PM | 5 PM | 6 PM | 7 PM | 8 PM | 9 PM | 10 PM | 11 PM | 12 AM+ | 1 AM+ | 2 AM+ | 3 AM+ |
| RECOT | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 |
| Eastern Daylight Time | 3 PM | 4 PM | 5 PM | 6 PM | 7 PM | 8 PM | 9 PM | 10 PM | 11 PM | 12 AM+ | 1 AM+ | 2 AM+ |
| Eastern Standard Time | 2 PM | 3 PM | 4 PM | 5 PM | 6 PM | 7 PM | 8 PM | 9 PM | 10 PM | 11 PM | 12 AM+ | 1 AM+ |
| Coordinated Universal Time (UTC) | 19 | 20 | 21 | 22 | 23 | 00+ | 01+ | 02+ | 03+ | 04+ | 05+ | 06+ |
| Japan Standard Time (JST) | 4 AM+ | 5 AM+ | 6 AM+ | 7 AM+ | 8 AM+ | 9 AM+ | 10 AM+ | 11 AM+ | 12 PM+ | 1 PM+ | 2 PM+ | 3 PM+ |
+ - next day.
User Agents used in REC FCC interfaces
Requests originating from REC Networks will come in on specific IP addresses (which are available to FCC staff on request) and in most cases, will include one of the following User-Agent strings in the request header (note: version numbers may vary based on development):
| LMSlist | Mozilla/5.0 (compatible; LMSlist/4.1; +https://recnet.com/eLMS-info) | (Formerly known as LMSbot) LMSlist interfaces with the LMS list of applications and then writes the information to the application lists and to the table that provides information for display on FCC.today. |
| eLMS | Mozilla/5.0 (compatible; eLMS/3.1; +https://recnet.com/eLMS-info) | For applications determined by LMSlist to be qualified for real-time processing, eLMS will retrieve the HTML of the application from LMS and then parse the information in the REC databases and for display in FCCdata.org. (Different version numbers of eLMS may appear) |
| FCC API | Mozilla/5.0 (compatible; FCCcontours/1.0; +https://recnet.com/eLMS-info) | Requests from this user agent are from various REC websites and tools, both external and internal to REC. We use the FCC Contours API to provide free government resources to our users, such as contour mapping data and transmitter site engineering information. While much of our mapping is cached and does not always go to the API, our more high volume public systems have been rate limited to reduce or eliminate traffic from bots and others who excessively use data. |
| EDOCS | Mozilla/5.0 (compatible; EDOCS/1.1; +https://recnet.com/eLMS-info) | Requests from REC to the FCC Content API for the retrieval of data related to electronic documents (EDOCS). This includes requests associated with news updates and broadcast applications public notice handling. |
| ECFS | Mozilla/5.0 (compatible; ECFS/1.1; +https://recnet.com/eLMS-info) | Requests from the REC website to view recent filings in the Electronic Comments Filing System. |
| OPIF | Mozilla/5.0 (compatible; OPIF/1.1; +https://recnet.com/eLMS-info) | Used to access the Online Public Inspection File API. Used primarily to verify the presence of a public file and the presence of a station logo. |
| PN | Mozilla/5.0 (compatible; eLMSPN/1.0; +https://recnet.com/eLMS-info) | Daily requests from REC to retrieve information related to the daily (Broadcast) 'Applications' public notice. |
When REC is retrieving a PDF for an authorization document, a more default user agent that does not appear above may be used.
These are not search engine bots, nor are they spiders (beyond the extent of LMSlist commanding eLMS to pick up an application detected). These tools only interface with the FCC and no other entity.
Routine scheduled operations throughout the day
eLMS
LMSlist will first access the public LMS list of applications. In the first transaction will access https://enterpriseefiling.fcc.gov/dataentry/public/tv/publicAppSearch.h… where it will pick up a session cookie and other coding from the source, then it will send a POST transaction to https://enterpriseefiling.fcc.gov/dataentry/public/tv/publicAppSearchRe…; LMSlist will then send another POST tranaction to increase the display size to 100 records. LMSlist will then page through all of the applications until it has gathered all of the application listings from the current business day and the previous business day. This generates a JSON file which is made available to LMSlist, which resides on a different server and has no direct contact with FCC resources. Therefore, on each scheduled run, LMSlist will send three calls to LMS to obtain information on the first 100 applications within the evaluation date range and then one additional call for each additional 100 applications within the evaluation date range (i.e. 300 applications = 5 calls).
LMSlist will then determine if we already have the data for the application on file, if we do not and the application is eligible for eLMS handling, then it will be queued for eLMS to gather the application details. eLMS will then send a GET transaction to LMS to obtain the HTML of a broadcast application. eLMS will then internally parse the HTML and create an application record in the REC database. On granted applications, eLMS will make a second call to download a PDF of the authorization document so that document can be parsed and added to the database. On applications involving engineering, eLMS will also send a GET transaction to the FCC Contours API in order to gather information on the application's service contour coordinates. Therefore:
- Non-engineering applications will get one call (LMS).
- Engineering pending applications will get two calls (LMS, Contour API).
- Engineering granted applications will get three calls (LMS, Contour API & LMS Authorization).
NOTE: Engineering applications which have one or more radials returning from the Contour API Coverage endpoint with a HAAT of 30 meters will result in a single additional call to the Contour API HAAT endpoint in order to obtain the accurate HAAT along each radial. This is a bug in the FCC Contour API that really needs to be fixed.
eLMS normally retrieves information on an application what it is filed, and when the status is changed (including to superceded in the event of an amendment)..
This routine operates on the following schedule:
| LMSlist run (RECOT) | eLMS processing (RECOT) | Frequency | |
| Monday-Sunday | 04:00 | Immediately after processing LMS data dump. | One time per day |
| Monday-Friday | 05:00~17:50 | 05:05~17:55 | Every 10 minutes |
| Monday-Friday | 18:00~21:00 | 18:05~21:05 | Every 30 minutes |
| Saturday-Sunday | 05:00~21:00 | 05:05~21:05 | Every 60 minutes |
| Everyday | 21:00~04:00 | No processing | None |
During eLMS processing of applications, the eLMS system will go into a locked state until the processing is complete. If the eLMS parser is called at a scheduled (or unscheduled) time and eLMS is still working from a previously scheduled time, eLMS will reject the subsequent request and that scheduled retrieval will be cancelled. eLMS will try again at the next scheduled time. This prevents the FCC LMS system from being hit with multiple simultaneous transactions initiated by different eLMS processing sessions.
Monday-Friday schedules will also operate on federal holidays.
The number of transactions may vary based on the application activity. If LMSlist does not detect any new or updated applications, no additional eLMS calls to the FCC LMS site will be made.
During the routine operation, eLMS enforces rate limiting towards the FCC LMS system through a mandatory sleep time of 5 seconds following the completion of the eLMS routines related to a broadcast application. This would mean that the average load from eLMS for application processing to the FCC LMS system is about 8 calls per minute.
Broadcast Applications Public Notice
On Monday through Friday (including federal holidays), REC will send a tranaction to the FCC Content API that will check for the presence of the day's Applications Public Notice. If the Applications Public Notice has been posted, the routine will make a second-call to the FCC Content API to get additional information on how to locate the specific document and then a third call that downloads the plain text version of the Notice. REC systems will than parse that information in order to update eLMS to indicate that a broadcast application has been accepted for filing.
This routine will start at 05:05 RECOT and will run each hour until either the Notice is released and the document has been downloaded and will run no later than 17:05 RECOT.
EDOCS
REC calls the FCC's Content API to obtain electronic document data (such as news releases, orders and notices). EDOCS does not download the actual PDF notices, only the meta-data from the API. This is done to build links to these items as they appear on FCC.today and other services. REC EDOCS service will run on the following schedule:
| RECOT | Frequency | |
| Daily | Prior to 05:10/05:30 | No service |
| Monday-Friday | 05:10~17:50 | Every 20 minutes |
| Saturday-Sunday | 05:30~17:30 | Every 60 minutes |
| Daily | After 17:50/17:30 | No service |
Filing window sanity checker
On 2-13-24, a new Sanity Checker process has been activated for the 2023 LPFM Filing Window. The Sanity Checker will run once per day during the morning routine to download LMS dump data. This application trues up our overall application data with window related data. If there is a discrepancy, we will do a call using the eLMS routine in order to refresh those applications. As of 2-13-24, this is currently at three applications. The sanity checker is designed to where there is more than 10 applications in this "lost" status, the automatic sanity checker will not run the eLMS routine but instead will provide notification to REC of an issue. We may elect to manually run these through eLMS again or identify another issue that may be causing these discrepancies.
Public Notice retrieval
This routine runs Monday through Friday, once per hour starting at 05:05 RECOT and no later than 17:05 RECOT. This routine will first use the EDOCS API to obtain the daily (broadcast) 'Applications' public notice and then once it is found, it will retrieve the plain text version of that document in order to parse it to update eLMS on applications that are accepted for filing. Once the program is successful in picking up the public notice, it will stop running for the day.
REC scheduled daily/weekly tasks to FCC resources
| Days | RECOT | Description of task |
| Sunday | 01:03 | ULS weekly data dump - Once per week, obtain six file dumps from ULS. (l_LMpriv.zip, l_LMbcast.zip, l_micro.zip, r_tower.zip, l_ship.zip, l_coast_zip). |
| Monday~Sunday | 01:12 | Census Update - This service creates an internal JSON file that includes information obtained through our internal Census database to provide a profile of the station's coverage. If the routine encounters an FM application that does not have full eLMS information, it will make calls to LMS and the Contours API (same as the eLMS related calls made during the day) to pick up the data to rebuild the record in eLMS. These calls to FCC resources should be more of a rare exception and not common. There is a built in 3 second pause per application to provide some rate limiting. |
| Monday | 01:17 | IBFS weekly data dump - Once per week, obtain one file (Bureaus/International/databases/IBFS.zip) from the FCC's FTP site. |
| Monday~Sunday | 03:45 | LMS Raw Data dump - Once per day, obtain the daily LMS dump file (Current_LMS_Dump.zip). This processing of the data from this dump starts at 04:05 RECOT. Immediately after this processing, an eLMS routine will be performed to obtain any applications filed between 17:00 the previous evening until 04:00 this morning. |
| Monday~Sunday | 08:55 | ULS Amateur Radio - Once per day, obtain the daily file dumps from ULS for Amateur Radio activity (2 files downloaded). a_am_XXX.zip & l_am_XXX.zip (XXX = day of week). |
| Monday~Friday | 18:35 | (temporarily disabled) LMS dismissal letter check - Resend eLMS transactions to each broadcast application that was dismissed during the business day to check to see if any additional application attachments (marked as internal) were added by staff in connection with a Media Bureau action. |
Rate limiting and bot prevention
REC has implemented a global rate limit on publicly accessible resources that call out to FCC Contour API resources as past abuse by third party bots resulted in excessive calls made to FCC resources from the REC network of systems. We have taken the following proactive steps to reduce calls to the FCC systems:
- We are limiting access to certain publicly accessible resources to no more than 6 hits per minute based on the first 7 characters in the string of an IP address. This pretty much puts the block at the Class B IP address level.
- On FCCdata, when a user agent has the string "bot/" or "Bytespider" in it, we are blocking access to the link to retrieve the REC HAAT tool. This is based on the trends in activity that were discovered in the enhanced access logging functionality that we recently put into place.
- On FCCdata, more than 10 accesses within a one minute period will return an HTTP error 403.
Attempts to access resources where rate limiting is applied or a 403 error is returned will count against the system limits as if they were otherwise permitted accesses. This means that any user or bot must maintain silence for at least one minute before they can re-access our resources. Accesses to FCCdata does not automatically result in a call to FCC resources, only when specific mapping applications are also called or if the HAAT link is also clicked (which is why we disabled access to that link for bots).
Internal requests made manually by a human at REC Networks are not rate limited.
Rate limits and operating schedules on the eLMS functionality that is automatically started by REC Networks are detailed in the information above.
Development and Repairs
During all hours (including business hours), REC may send one-off requests through eLMS to obtain applications for the purpose of developing new eLMS functionality or to send a "manual reset" of an application in eLMS that may have been mis-parsed or had a bad contour map due to the failure of the FCC Contour API.
Special Projects
Special projects involve the need to have eLMS obtain a large number of pre-identified applications. This is done in order to address specific issues or to migrate new application types into eLMS handling.
Normally, special projects will normally only be conducted between Friday at 6PM EDT and Monday at 7AM EDT as to use the systems during periods of predicted low demand. In case of a major operational issue that cannot wait until the weekend, REC will limit special projects to between 6PM and 7AM on any day of the week or any time on the weekends.
Sometimes, special projects are intentionally timed for federal holiday weekends if it is necessary to obtain the applications we need.
During special projects, requests to the FCC's LMS system are normally rate limited to one request every 10 seconds (6 per minute) (on granted applications, one "request" may involve two calls to LMS to get the application and authorization document), but in no case shall special projects generate requests less than every 5 seconds apart (12 per minute). We are cognizant of the needs of the LMS server.
REC Networks public websites
REC Networks operates various websites that use data gathered from FCC resources. Most of these services are free of charge for use. The following information describes the website and how FCC resources are used.
FCCdata
FCCdata is the most comprehensive website for inquiries into broadcast facilities, amateur radio and other services. FCCdata also includes broadcast and in some cases, amateur radio information from Canada, Mexico, Australia, Japan, United Kingdom and Ireland. FCCdata is unique from other broadcast query websites (including those at the FCC) because it can display full FM facility data, usually within 20 minutes of the filing of the application. The same timeline exists for non-engineering applications from all LMS supported broadcast services for the display of application information.
FCC broadcast data is derived from the data gathered in our eLMS routines as well as from daily data dumps from LMS, ULS and IBFS. Site usage may result in calls to the OPIF API and to the FCC Contours API. Users are rate limited where if they make more than 6 accesses per minute, they will be restricted from automatically receiving FCC resources. More than 10 accesses per minute will block them from the site.
As of 2-14-2024, FCCdata averages about 4.7K page views per week.
fcc.today
FCC.today operates as a “dashboard” for application and action activity in the Audio and Video Divisions of the Media Bureau. FCC.today uses data collected through the REC LMSlist routine to provide “near real time” updates on applications and actions that are processed through LMS. FCC.today also displays information on recent FCC releases that involve the Media Bureau, FCC daily business items, emergency alerts received through IPAWS OPEN, information on broadcast pleadings as well as DC federal employee reporting status. It is also the home of “fcc.today – the podcast”.
Public website usage on FCC.today does not create any direct calls to FCC resources. All data shown on FCC.today comes from the various scheduled background processes that are described above. This includes scheduled data gathering routines using our LMSlist and EDOCS processes and the LMS daily dump file as well as data received through the API at the Office of Personnel Management and through our memorandum of understanding with the Federal Emergency Management Agency.
As of 2-14-2024, FCC.today averages about 3.8K page views per week.
REC Main Website
REC’s main website is a comprehensive Drupal-based platform which is used for both public websites as well as walled garden applications for internal use only. Data used can range across our entire universe of stored databases across multiple federal agencies. Some tools replicate FCC tools that utilize JavaScript such as Curves and FM-Model.
Some applications on the REC Main Website may generate user requested calls to FCC resources. Most of these applications have been rate limited, mainly to 6 requests per minute. The most frequently used of such tools is:
HAAT/Elevation/ERP Tool
This tool makes calls to the FCC Contours API to provide an improved display of height above average terrain, site elevations and calculate effective radiated power. Also obtains data from the US Geological Survey and private sector data providers.
As of 2-14-2024, recnet.com gets an average of about 2.8K page views per week. We have no specific measurement for the HAAT Tool.
myLPFM
myLPFM is a password protected system that is limited in access to broadcast stations in the Low Power FM Radio Service. myLPFM allows LPFM stations to perform station management functions, manage their visibility on station directories provided by both REC and third party providers, provide services that will assist in filing modification applications, obtain emergency alert information, engineering information for HD Radio upgrades and assistance with interference from subsequently authorized FM translator stations as well as information on potential encroachment from any subsequently applied for facility.
Various features within myLPFM will make user-requested calls to the FCC Contours API to get various information necessary for the different sub-applications within myLPFM. Some data received through the Contours API is cached for future access thus speeding up the user experience and eliminating unnecessary traffic to the FCC API.
myLPFM experiences a very low usage level.
Contact Information
In the event of issues with REC Networks' automated access to the FCC systems, please first contact Michelle Bradley, CBT via email at lpfm@recnet.com or at 202 621-2355.
More information about eLMS available at https://recnet.com/eLMS
As stated. REC's need to access these public FCC resources are for business, research and public dissemination purposes, consistent with 44 USC §3501 et seq. and therefore, is not malicious or with other ill-intent.
REC does not have any automated resources that accesses FCC resources behind a password. We have humans to do that at the request of our clients.
Thank you for your understanding.
