The company behind the Bacup housing development has accused some objectors of violence, damage and trespass, and issued a photo of a worker it says was hit with a wooden stake.
Other recent photos on social media appear to show groups of people on the land.
Residents' views appear divided about recent events: one neighbour said things have become 'way out-of-hand', raising safety fears for children and site workers but another highlighted residents' concerns, saying the situation will take a long time to recover.
Seventy-one homes are planned on land between Fieldfare Way, Goldcrest Avenue and Pennine Road, known as Back Meadow.
The plan by McDermott Homes, which began in 2019, was opposed locally and by Rossendale councillors nut permission was ultimately granted at a national planning appeal earlier this year.
Work has started but a number of incidents have reportedly taken place.
Police attended one on September 4 and are now looking at a reported assault this week, September 16.
Separately, there are accusations from objectors that planning conditions are not being followed but these have been strongly rejected by McDermott Homes.
A spokesperson for McDermott told the Local Democracy Reporting Service: "We have engaged with Rossendale Council's planning department throughout the process and, after further discussions with the officers last week, we have a clear understanding of the extent of pre-development works permitted to carry out. This is extremely important at this site, given the extent of objection and negative sentiment of the residents against any development on this land.
"Members of the council's planning enforcement team have been on the site this week and are happy with the measures taken and satisfied that the work done so far has not breached any planning conditions pursuant to the appeal decision obtained."
The spokesperson added: "Some residents are continually seeking ways to obstruct any site work, every step of the way. The security fencing at the access points has finally been erected and signs posted. Yet residents continue to trespass on the site and assault contractors.
"A member of the security team was seriously assaulted this week and this has been reported to the police. The picture shows a security guard who was attacked from the side with a wooden stake. It goes to show the extent of violence some residents will go to.
"Again, we must reiterate that no planning condition has been breached at any stage. The only people who are in breach of anything is the few local objectors, members of a local Facebook group with their anti-social behaviour antics which amount to violence, now aggravated trespass and criminal damage to perimeter fencing.
"We remain committed to delivering this housing allocation put forward by Rossendale Council and providing the area with much-needed housing to meet their housing targets."
A Lancashire Police spokesperson said: "We were called at 4.50pm on September 16 to a construction site on Fieldfare Way, to a report of a nuisance being caused. It was reported that a group of children had gathered and were refusing to leave the site. A report was made that a worker at the site had been assaulted. Enquiries are continuing. Anyone with information, please contact 101 quoting log 0910 of September 16th."
One resident, who did not want to be named, said: "I live nearby the proposed housing site. Whilst I do not agree with the housing proposals, I feel it is all getting way out-of-hand. I worry for the safety of children and workmen on the site, due to the measures that some other residents are going to.
"There have been some comments on social media to incite violence and encourage others to cause damage. This has escalated. I fear for children if they are being encouraged to run around the machines on the site and fight with security staff on site."
Another resident, Janet Boon, said: "Some fencing was put up but then some of it was pulled down, with some people unhappy about access issues. There are also concerns about the diggers working on the land, about trees, badgers, deer and bats. It looks like they've just come in and bulldozed the lot, from what I know. People are worried that trees are being taken down that shouldn't. That planning conditions aren't being followed."
Rossendale Council planning committee's decision to refuse the McDermott Homes proposal went against borough planning officers' recommendation to approve it. Although councillors had a right to take their own decision, their reasons for refusal, about highway safety and bio-diversity impact, were not satisfactory, a national inspector ruled earlier this year. Detailed evidence was needed to support their road safety argument while McDermott Homes had addressed wildlife topics, the inspector said.
In recent months, McDermott Homes has submitted various smaller planning application to Rossendale Council, seeking approval for details around elements such roads and sewers, a construction environmental management plan, building materials and work phasing.
Janet added: "Many residents have been against this plan for a long time. It was a natural space for years with woodland and wildlife. Dog walkers used it and it was well-used in the covid pandemic. Kids had a BMX area for bikes, which was used for years. Some people say kids could use another cycle track at Stubbylee Park but that's not close to these homes and getting to Stubbylee involves using a winding road."
Asked what a way forward might be, the Bacup resident responded: "I don't know. There are a few issues that people are unhappy with. I think the situation will take a long time to recover. People feel the whole process was not handled well.
"There was public consultation but people feel they weren't really listened to. We wrote to everybody including the Woodland Trust. We went to Rossendale Council planning meetings with prudent arguments not NIMBY comments. The plan eventually went to an appeal, which was like a big government meeting. As far as I know, residents weren't aware they could attend."
Regarding wildlife, deer may have moved elsewhere, according to some social media posts.